The disadvantage of a built-in battery is that you have to
a) Be very careful not to expose the player to extreme conditions , which ruin the battery , and
b) Pay big money for the battery if it ever happens to fail
If you don't want to keep buying batteries , then another option is buying rechargable AA batteries and buying a battery charger . I implore you to consider this option - this way , you never have to waste time recharging your batteries as you can always have one spare battery ready to be fit into the player , whereas with a built-in you have to charge it , which takes time . And the cost of getting a new battery is almost negligible if something bad happens to the battery in case they are replaceable , as they are in the case of the iAudio G3 .
The price of a 2 GB iPod is 200 $ . The cost of the comparable iAudio G3 2 GB player is 140 $ . The cost of the Toshiba AA and AAA battery recharger is around 19 $ including shipping . The cost of a pack of 24 rechargable batteries is around 25 $ .
Total cost of iPod : 200 $
Total cost of iAudio + Battery recharger + pack of 24 AA batteries : 184 $ .
Your choice

.
And note that the hard-disk based model uses an in-built Li-ion battery , if that is what you want , but as you said that you were going jogging , it doesn't seem such a good choice .
Even if you choose to go with another vendor , make sure that it at least plays the ogg and wav formats - ogg is great for ripped music , much better than mp3 . And one of the the best benefits is that it is impossible to DRM something in ogg , as you can with mp3 . Plus , of course , ogg is a free software format .