if you have any ideas for improvements, i'm all ears
You are going to regret this! Hehe.
Use two tactical layers: Offense and Defense. The Offense layer works pretty much as now. The Defense layer will work on potential threats. Each layer will ask for some tasks to be fulfilled, with a priority level, and unit type/strength requirements. When a unit is set to a task, it will keep doing the same task until another task with higher priority demands the unit, or the task is completed. Being closer to the tile with a task increases the probability of that unit to be selected for the task (this means that if an extra tercio is needed for a task, it will take the nearest tercio that works on a lower priority task, or else the nearest idle tercio unit).
The Defense layer identifies good defensive points, weak points and sentry points within our territory boundaries, including nearby no-man's land. A good defensive point has rivers, a nearby hill in our side, open terrain in the other side, fast healing from owned territory, good road connection and fewer tiles that allow access to enemy units (with lakes and/or mountains, except if Incas/Songhai are neighbours). Cities can be defense points with extra defense, healing and attack, but having improved tiles ahead counts against it. For naval, defensive tiles are always coastal cities. A weak point is a tile located in the empire boundary, near enemy lands, with no defensive points nearer than four pathfinding steps. Weak points are a place for gathering idle units, place them in defensive formation and set workers to improve defense with higher priority. Sentry points are linked to boundary points, where there is fog of war in unclaimed territory.
Defense layer will label the tiles in the boundary as defensive or weak, spacing them each four pathfinding steps across the boundary, and will set a priority. Priority is decided upon the closeness to aggressive neighbours, the value of the city and resources it directly defends and the closeness of defensive resources (if a nearby defense point can send reinforcements in fewer than two turns, or a populated city is nearby). Sentry points are related to a defense or weak point, and its priority is inherited from the point it is watching for.
1. DEFENSE POINTS. A defensive task may change depending on the era and terrain, but it usually consists on a melee unit backed by a ranged unit. If there is no terrain bonuses in the defensive tile, it can be a mounted melee instead. And a siege unit can be called for if no ranged unit is available.
2. WEAK POINTS. A weak point needs more units to fulfill its defensive role. All units not assigned to a task will gather at weak points and stay in defensive formation.
3. SENTRY POINTS. Sentry tasks are fulfilled by scouts, or else mounted units. If there are two sentry points that the scout can visit in two turns, then the scout will receive the patrol task between those two points.
PREPARATION TASKS
After Defense, Weak and Sentry Points are identificated and priorized, units have to be assigned. If the empire is at peace, all units not working on an anti-barb squad or garrison can be used for defense. Each point will ask for units, starting by those with highest priority, and the closest matching units will go to the tile (or around it). After the unit has arrived to point, the priority of nearby points will be reevaluated, since there is now an extra nearby military unit. Garrisons per se have low priority and these units may be used for attacks or reinforcements. If possible, cities that are used as defense points will focus on defense buildings.
New cities or new territory around boundary points will also cause them to reevaluate boundary point locations.
A worker squad will take care of defense improvements for each boundary point (clearing forests ahead, connecting to road network, building forts), beginning by the weakest points with highest priority. If there weren't enough military units of the required type to fulfill all defensive tasks, then it will increase the will for that kind of units.
THREAT LEVEL
In case we spot other civs (neutral) military units moving, we raise the threat level. Just for caution, we increase the priority for tasks near the spotted units. This means any nearby boundary point is going to receive extra units and workers are prioritizing roads here. The priority raise will depend on the relative strength of the shown units. We don't want to mobilize tercios for a single foreign warrior, but we do want it for a couple of knights.
If war is declared, then it's not just for caution. An attack is expected, and we need to estimate how long the point is going to hold. Then compare it with the time it will take for reinforcements to arrive. If help cannot arrive in time, then a retreat task is in order. Retreating is a complex task of finding the next defensible position and moving all the units there while minimizing losts.
In both cases, only enough units to fulfill the tasks should be employed. Idle units without any task may join combat, though.
So, basically the Defense layer use a set of fixed tiles near the borders (which may include cities), says which kind of tasks are suitable there (minimal defense, units gathering or scouting) and decides a priority level depending on tactical factors (strenght of nearby friendly, neutral and enemy military units, proximity to foreign/enemy territory). Meanwhile the Offense layer keeps launching surprise attacks by several ways if possible.
When at war, the Offense layer may control up to 50% of the units. A warmonger may allow the Offense layer to control up to 80% of the army, while a peacemonger may limit that to a 30%, for example.