The problem with looking at it from the point of view of sets, is that point of view presupposed the concept of knowable cardinality and therefore all finite numbers. Yes that approach will get you infinity from one, but does not show that you need to do it that way.Yet a concept of 'many things' also means 'one many things' and not 'indistinct if many is many or non-many'. Isn't anything that is conceptualised as being something, viewed as being something because it is set as One?
*
Looking at the above from the point of view of sets: If a set has 'many things', but there was axiomatically no set of 'One thing', then what exactly does a set with many things even mean, and what is it juxtaposed to?
You seem to argue that the antithesis here is between indistinct 'many' and 'infinite', but 'many' is closer to 'infinite' from most points of view than 'one' is, and thus is closer to being the same type/category. Thus it is not practical to use it as the primary juxtaposition, cause one juxtaposes opposites as a basis, and One---Infinite is more of a polarity and of two ends.
I'm not talking about the concept of "many" at all, I'm talking about the concept of "more". You don't need all of set theory to have that concept. You need a quantity that is order-comparable, without an upper bound. You don't need to be able to say that one quantity is exactly so much more than another.
Take for example water. You don't need to imagine an atom of water to imagine an infinite ocean. But for "infinite" to describe a quantity of water you do need to be able to compare it to smaller bodies like a puddle and a lake. A puddle is not "one" water". But a if you realize that a lake is bigger than a puddle, you can imagine an even bigger lake. Ad infinitum.