Oh, so you are just talking about something being infinite in one of the dimensions - whether it's time or space? Then yeah, we might never know. But the mathematical concept of infinity doesn't really rest on us ever finding out; it's just a quite natural concept that would have arisen one way or another.
It's easy to go from "bounded" to "unbounded", you've just got to think a bit and there you go.
I once wrote a short story where a successor-race to humans was not having a theoretical concept of infinity. To them it was a practical concept, given their world was very small and bounded by a massive free-fall/chasm, which they sensed as infinite in some manner.

I think that humans are very much defined as thinkers by the ability to have notions like (seemingly) very defined (eg One, in some level, in others it is not that defined), and very vague (eg Infinity).