I think that the functioning of our universe operates in line with this rule-set we call 'mathematics' - but mathematics is not actually the rules. Mathematics is more like a 'feature' of the existence of things that have a definite existence, and is more or less inescapable. Imagine a conceptual universe that only contains one thing, there is not even any space for the thing to exist in, since this creates a dichotomy - we have two things, the thing itself and the space it is present within. Now assign it an attribute - oh look we have two things again, the thing we started with and the attribute it has.
It is often said that physics is applied maths, but I think this is incorrect. It is easy to point at chemistry for example and say 'that's just advanced electrostatics', but what is there in physics that is actually derived from maths in a similar way? (possibly stochastic i.e. quantum phenomena, but i'm not sure how the argument would go).
Is the maths we know fundamental, or are other systems of 'keeping account' possible? Non-diophantine mathematics is a thing, with real-world applications. Maybe other universes are possible: imagine a sort of positive energy universe in which one plus one makes 2.5 - the action of adding things makes more than you started with.