...The major issue I'm running into is AI military. Even on emperor I was able to easily keep up with the AI military, now I can spam military constantly in every city and remain in last place...
This is your mistake.
You can't keep up, your wasting gold and hammers even trying to be in the middle. Unless you are dead last in military for the early game you are doing something very wrong. You might move up to #6 if you have a very early successful war.
You will need to focus on when you think you will need a military tech, and when any given AI is likely to DoW. It's mostly a matter of making good predictions 10-20 turns ahead of time. You can do this because you made it through emperor level and are able to predict when you need to start improving resource, this is much the same.
On immortal the early DoW comes ~T50 and you can fend off with archers, it comes around ~T35 on deity. The next DoW comes around T100 on immortal and T80 on deity, and you need comp bows to fend it off. You should be in a tech position to be able to get the needed tech if you get DoW'd. I usually won't research construction until it comes up naturally in my tech progress, but if I have a scary neighbor, I will take some turns off of it so I am more ready.
The AI gets a big advantage over you in everything from happiness, production, and starting units. Getting parity tech wise is my first priority.