I'm not sure what to do with the information, exactly, but here is a spreadsheet which may help in redesigning the late game tech tree. It shows some interesting statistics about each tech in the tech tree. For each tech, it lists:
* Name and era
* Current X position on the tech chooser screen
* Min possible X position, assuming we avoid any backwards arrows. So if a tech has prerequisites at X=2, X=2, X=4 then its own minimum X is 5. Formally, find all of the prerequisite techs, take the max of their X values, and add one.
* Max possible X position, assuming we do not extend the tree past the end. Future tech is at X=18. Suppose there is some tech at X=12, however, the only thing that tech is a prerequisite for is Future Tech. Then we could slide that tech anywhere along the X axis as far as X=17 without introducing any backwards arrows. Formally, find all the techs of which this is a prereq, take the min of their X values, and subtract one.
* NumSets: not interesting to you. This is the number of actually different ways you can reach this tech. For example Divine Mandate has two different ways, because one of its prereqs is Feudalism, and either Fanaticism or Education leads to Feudalism. I will wait here while you list out the 18 different paths you can use to get Weirding Way.
* MinTechs: the minimum number of other techs you must have, in order to reach this tech.
* Cost: the base cost of this tech by itself
* CumCost: the total beaker cost of researching this tech, including all its prereqs.
This immediately exposed some bugs which were caused by moving techs around incautiously (probably by me). Each era should contain techs with similar X positions: ancient is X=1 through X=3, classical is X=4 through X=6, etc. The homeworld screen allows you to purchase any unit of the same era you are in. But, about 8 techs had the wrong era. For example, Planetary Ecology (X=6) was set to modern era (X should be >= 15). This would allow you to research Planetary Ecology at relatively low cost, and get access to advanced units through the homeworld screen. I have cleaned this up for 1.7.1.
I figure that the cumulative cost is what we should focus on, although I am not sure exactly what to do with it. You can see that the cumulative cost of techs at a particular level varies widely. 2:1 cost at the same level is not uncommon, and 3:1 happens occasionally. For example, Mind Training (4K cumulative cost, 10 prereq techs) and Solid Fuel (12K and 29) are at the same level.
The program I wrote to generate this table is available at
this link. Unzip the executable into your assets/xml/technologies directory and execute it; it will write a file called tech-costs.csv into the same directory. If you do not have write access into your program files directories, copy the civ4techinfos.xml file from there into some directory where you do have write access. One warning, however, the program is not guaranteed to work for all tech trees, and I did not put much work into nice error messages. In particular I tried it on the vanilla BTS tech tree and it just runs forever until it uses up all your memory. Anyway, if you want to try some small changes to the DW tech tree and see the effect, it will probably work.