First instance of piracy

[UWHabs]

That is true. What I wanted to point out is that LLMs are not completely deterministic, in the sense that they do not always do exactly the same thing on the same prompt. This is especially the case if a specific prompt becomes a part of other instructions. The LLM may interpret a specific part of a prompt differently depending on the context. That is what I tried to illustrate with the example of 5 + 5. The LLM may be able to give the correct answer of 10 in most cases, but in some contexts it may differ from that answer.

This may not be a defining property, but as far as I can tell almost all of the successful LLMs show this non-deterministic behaviour. The inherent cause for this may be deterministic in itself, in the sense that you can figure out why the model acted differently than expected in a specific context. But for most users, this may remain mysterious, so I think you always have to treat AI as having some inherent unpredictability. This is different from a calculator, which will always compute 5 + 5 = 10, even if you input this in a sequence of various other inputs.

LLMs use the randomness to avoid getting stuck in one spot.

Like, for example, if you programmed a civ AI LLM without randomness, it would probably get stuck where it always settles its founder in place on turn 1, it always builds a scout first, it always researches Animal Husbandry first, etc... I mean, maybe in the end that is the optimal configuration, but especially in the learning stages, it needs a little randomness in order to explore if the warrior-first play might make sense, and so on. You don't want so much randomness that the founder takes 30 turns to settle, sure.

And @Arcaian has some great points in their post here too. Just because some people here want an AI that is better and puts up a bigger fight, you don't necessarily want the AI to actually be perfect. Because a perfect AI will be ruthless, and probably won't play thematically. Sure you hope the game design gives enough bonuses for Egypt to always want to settle on navigable rivers, but a "perfect" AI might ignore that for some other reason (like aggressive forward settling, or stuff like that). Frankly for a game as big and complex as civ, perfect is impossible. What I think deep down people want is "not bad". In that, I want the AI to position troops in range of their commanders, to not walk settlers un-escorted around the map, to coordinate their units to flank you, to not just start walking catapults across a lake, etc... I want them to actually expand to claim resources, not put a bunch of farms out and ignore the gold tile. I'd like them to actually plan their adjacencies, and put specialists in the right spots, so that they can actually compete in the exploration era.

You don't an LLM for most of that. But the big problem with a big complex game like civ is that most if not all of that isn't really hard-coded. The AI doesn't know what a granary is, they just know it's a TYPE_WAREHOUSE building that gives YIELD_FOOD, etc... Even if they basically had the AI put out internal map tacks for where they wanted every building to go, they would get stuck I'm sure because it's super complicated to figure out what order to place things. When do you take the +2 adjacency library right now, vs getting the +3 adjacency library that you need a few buildings to jump over to reach that other tile?