Layden Jar's are capacitors and not batteries.
The distinction being that batteries* store chemical potential energy that is converted to electrical energy via electrochemical reactions. Capacitors store energy in an electric field. And while they have some definite similarities in that they both store energy they are quite distinct pieces of technology.
*I'm using the common parlance usage of the term battery. It gets a bit messy when we get into historical and technical usages. For instance Ben Franklin invented a type of capacitor (that wasn't Layden Jar) that he called an 'Electric Battery' and him calling it a battery is why we call batteries batteries today even if it fails that common parlance term.
The Baghdad Battery is speculated to be a proper battery (again in the common usage) whether it actually was or not is rather controversial. Fun one to think about, definitely something I'm not sure about.