There were new civs added to the base game: in PTW, and again in Conquests. Civ3 predates this current model of endless DLC and continuous development. Back then games were largely one-and-done affairs, and if they were commercially successful they might have an expansion pack or two released in physical stores. Even post-release patches were hard to deliver; you couldn't just assume the player would have an internet connection or be willing to download a large file. So in that era there were very limited opportunities to add content, but Civ3 was successful and did.
As to why mods don't just add more, as mentioned that's "hard coded". Games, like computers, were less advanced back then and often programmed with just enough to do the job. Almost every kind of thing in Civ3 is limited to some power of 2, depending on the size of the number (number of bits) used to keep track of it. Even the developers would have had quite a bit of work to change these limits, which would affect the layout of data in saves and scenario files. So 32 civs (including barbarians) is all that it can handle. The good news is that Civ3 has an extensive mod library with hundreds of custom civs in different combinations available. You just have to play one mod at a time, instead of plugging them into the base game.