Ok, a bit more messing around and I think I've actually got the cause narrowed down quite a bit. Apparently it's triggered purely by the presence of Scoundrels in the city. Basically, if I take a stack of Scoundrels and park them in the city, the next turn a large stack of criminals will spawn. I'm not 100% sure of this yet, but it looks as though the number of resulting criminals is always equal or greater to the size of the stack of Scoundrels. So far I've tried with a stack of 1, 21, and 22 Scoundrels. 1 Scoundrel did not cause any spawning, but of note that single Scoundrel did not have the Wanted promotion initially. The stack of 21, all Wanted, caused a stack of 31 assorted criminals to spawn. 22 Scoundrels, all Wanted, resulted in 32 spawned criminals. I'll try a couple of different stack sizes and see what happens.
The faulty AI might be a false lead, or an separate issue. In these past two spawns not a single criminal has gone pillaging or attacked/suicided on city defenders/police. Also, the "extra" units spawned in the previous, positive-crime tests might be the result of Scoundrels being among the spawned criminals then causing extra spawns.
edit: Update on the testing, parking 10 Scoundrels resulted in zero criminals spawning. 12 resulted in 24 criminal spawns. 15 resulted in 32 criminals spawning. Not sure if it's due to random chance, or if there's some sort of critical mass of Scoundrels to trigger spawning, or there's some other oddball factor involved. Also tried with 15 Thugs, same results. 24 Scoundrels, 12 Hajduks (Culture version of Scoundrel), and 15 Thugs all at once resulted in 32 criminals.
edit 2: Skip forward a few dozen turns, and just the 24 Scoundrels and 12 Hajduks has been consistently spawning 36 per turn, as long as I arrest all of the spawned criminals on the same turn. I've been dancing the criminals in and out of my city every other turn to trigger a spawn, arrest them, then give my Police Mechs a turn to heal before repeating. Slave farming made easy!
edit 3: Looking at CvCity.cpp, I think I found what's allowing them to spawn both past the limit and with negative crime. The relevant section of code. iCurrentValue is, if I'm reading this correctly, the crime property value in the city. I'm not familiar with the structure of the rest of the code, but I don't believe there's a check on whether Crime is positive or negative before the adjustment for iNumCriminals.
Code:
{
iCurrentValue -= ((iCurrentValue/10) * iNumCriminals);
}
iCurrentValue = std::max(0,iCurrentValue);
Thus, what is happening:
When there's no criminals in the city, iNumCriminals is 0, iCurrentValue is unchanged by the -=, and the second line then zeroes iCurrentValue if it was negative, or leaves it alone if not. No issues here.
When there are more than 10 criminals present in the city but iCurrentValue is negative, we wind up subtracting a negative value of magnitude greater than that of iCurrentValue from it, thus resulting in a positive value. Example, crime is -1000, 15 criminals in the city. The calculation becomes -1000 - ((-1000/10)*15) = -1000 - (-1500) = +500. Thus, we have a negative crime value being flipped to a positive value and passed to the spawning code, while at the same time bypassing the limit.
I can see several potential fixes for this, not sure what would work best in terms of performance or other issues, I'm not actually a programmer. We could simply use a conditional to bypass the entire block of criminal spawn code when crime is negative to begin with. We could rewrite the reduction-per-criminal calculation to multiply instead of decrement (something along the lines of iCurrentValueAdjusted = iCurrentValue * max(0,((iMaxCriminals - iNumCriminals)/iMaxCriminals)) where iMaxCriminals is however many criminals you want the cap at). We could rewrite it to enforce a maximum effective value on iNumCriminals (iCurrentValue -= ((iCurrentValue/iMaxCriminals) * (min(iNumCriminals,iMaxCriminals))
Note that the above is just pseudocode, not actual code, since I don't know C++. Also, I used iMaxCriminals so that if anything crops up in the future or changes are made, we don't all wind up wondering what this mysterious "10" value is meant to be or where it came from, and to make it easier to tweak.
I'm not sure what was allowing it to overshoot in the positive-crime cases, though.
Unrelated issues:
1) The Mormon LDS Missionary Training Center National Wonder removes access to Coffee, but not to Roasted Coffee. Thus, I believe it blocks only the construction of vicinity buildings (if any) and the Roasting House/Factory, but not the Coffee House or Cafe.
2) Biopunk unlocks a building that produces Organic Computers, but Organic Computers as a resource isn't otherwise unlocked until much later in the tech tree, at DNA Computing. This makes the one from Biopunk useless. If the intent was for Biopunk to provide much earlier access to them as a benefit, the reveal/enable needs to be moved forward. What happens if a resource's reveal/enable are set twice?
3) The Vertical Farm improvement is an upgrade to Farms, Plantations, and Pastures. However, several Plantation and Pasture resources aren't provided by Vertical Farms, so the upgrade cuts off the resource access. This includes Llama, Camel, and Incense. Almost certainly others as well, but I haven't gotten around to checking.
4) Extraction Facilities, the unit, display as a Modern Workboat. Not sure if the art assets exist to have it display the same as the improvement, but if not, then changing it to the same graphics as a Construction Ship would be better at least.