DLL development discussions

So separation isnt possible..sadly.
Then as in AXXXE thread disscused, engine developing will be too difficult task. How hard to make
very extensive asynchronous proxy layer between the DLL game model and the APIs to/from the main engine
?
and do i understand correctly, that in such way bts engine will be used only as renderer?
 
To a very large extent that means you can only run the non-engine evaluation code in windows when the engine thinks it should be running (i.e. - while the engine thread is in CvGame::update()).
Hmm, but maybe there are things, which can be evaluated even when the main thread is out of CvGame::update() and we would only fetch on the update() call. It would even be able to move some evaluation to the player's turn time, if we would find appropriate data to evaluate.
 
Hmm, but maybe there are things, which can be evaluated even when the main thread is out of CvGame::update() and we would only fetch on the update() call. It would even be able to move some evaluation to the player's turn time, if we would find appropriate data to evaluate.

In principal there are, but it's no low-hanging fruit. For example we could run speculative tech evaluations so that they were pre-cached, or speculative battle simulations, or ...

The problem is that the hit rate on use of most of the calculated results would be quite low (at least without significant changes to the AI structure), so while we'd be doping the calculations off the main thread, much of the work would actually wind up being thrown away. Since loading one core decreases the performance of others on most modern CPUs (due to limited turbo opportunities), and increases energy consumption (lowers battery life if on a laptop), it's unclear ho worthwhile it would be. There probably ARE useful calculations we could make, but chooses which is not obvious.
 
Then as in AXXXE thread disscused, engine developing will be too difficult task.
Too time consuming, probably. But it is not so difficult, for such game like this one. With almost flat world, no physics simulation and 3'rd person view, there is hell less problem we need to worry about, then for example with 1'st-person open-area shooter engine.

and do i understand correctly, that in such way bts engine will be used only as renderer?
I think it is already used mostly as a renderer.

About this proxy, it depends on what api is provided by the engine. But it's the question for Koshling.
 
Any idea about multi-thread testing other then brute-force, that is running a million times each two possibly conflicting operations?
 
Any idea about multi-thread testing other then brute-force, that is running a million times each two possibly conflicting operations?

No. I am proceeding by trying to keep the parallel sections reasonably controlled (so that I can more or less see by inspection the set of descendant methods that are the nodes of a spanning tree of all possible call graphs from within the parallel section). If that set is reasonably small then again by inspection I can have a fair idea of what will potentially need locks and add them proactively. Admittedly this is not truely robust because the sets tend to be too large to have complete confidence. However, from there I just work on the principle that this isn't software for running a nuclear reactor, and that I will prioritize trying to pin down any issues that arise when they do.

This approach worked well enough for the parallelization of the city update, so I plan to continue with that model. There will be bugs from time to time, but all I can say is that I will attempt to fix them as they occur.

Nothing ventured, nothing gained. However if you have good ideas for more detailed testing feel fee to suggest. In some specific case I have added unit tests - the parallel pathing engine has a swlt test mode that generates thousands of random paths and compares to a reference implementation (non parallel) for example, which I used a fair bit while developing it.
 
@Koshling, I know you are rather using an abstraction layer, but who knows what future will bring? So I though, I will inform you the fence CPU instructions can be used to "atomize" things only on single-processor single-core machines. Thus, the advice you gave me is not actual for some time. ;)

About testing "paralization", the only thing is the brute-force I mentioned. -- Just call very many times each pair of possibly conflicting functions in two threads, with running a small random number of NOP operations before each call. -- But I guess, I can live with "not-a-nuclear-reactor approach" here. :P
 
Eh, I am losing my strength. CPUs are so non-atomic, that you can get crazy when trying to avoid interlocked operations. It looks like I will need to make the pointers to stored object to be two-directions linked lists and 3 or 4 times fatter then they should be, just because I can do incrementation and decrementation without interlocks. Really this non-atomicity is insane. :hammer2:

Sorry, I really needed to complain.
 
Unfortunately, I will not have time to help with C2C now. I have been completely unaware, how sh*t are current CPUs for parallel evaluation on the same piece of memory. And now I will need to learn about few thing to try to rescue my data structure for multi-treading without lost of performance. Together with my other work, it will probably take enough time, to allow me eat and maybe sleep once in few days, so contributing code writing is out of order now. Hope you don't mind.
 
@Koshling, I know you are rather using an abstraction layer, but who knows what future will bring? So I though, I will inform you the fence CPU instructions can be used to "atomize" things only on single-processor single-core machines. Thus, the advice you gave me is not actual for some time. ;)

About testing "paralization", the only thing is the brute-force I mentioned. -- Just call very many times each pair of possibly conflicting functions in two threads, with running a small random number of NOP operations before each call. -- But I guess, I can live with "not-a-nuclear-reactor approach" here. :P

I am almost totally confident your statement is incorrect. Please provide source references.
 
Maybe you are too confident? I could misunderstood something. But according to Intel's documentation fences have the descriptions like this
Performs a serializing operation on all load-from-memory and store-to-memory instructions that were issued prior the MFENCE instruction. This serializing operation guarantees that every load and store instruction that precedes the MFENCE instruction in program order becomes globally visible before any load or store instruction that follows the MFENCE instruction.
And this is not enough to guarantee atomicity, unless there is some other mechanism, which I am not aware.

If you have 0 in memory [addr] and run the following code on two processors, according to the specification, there is nothing which prevent from reading 0 by both processors.

Code:
BTS WORD PTR [addr], 0
MFENCE
 
Maybe you are too confident? I could misunderstood something. But according to Intel's documentation fences have the descriptions like this

And this is not enough to guarantee atomicity, unless there is some other mechanism, which I am not aware.

If you have 0 in memory [addr] and run the following code on two processors, according to the specification, there is nothing which prevent from reading 0 by both processors.

Code:
BTS WORD PTR [addr], 0
MFENCE

That is correct, but you can apply a luck modifier to the memory access, which guarantees that other memory accesses to the same location do not occur concurrent and acts as an implicit fence as a (necessary) side effect. It is the lock modifier that is crucial her - it's the fence side effect that causes (most of) the performance hit

Edit - see http://home.deib.polimi.it/speziale/tech/memory_consistency/mc.html for instance
 
Can we edit the element that controls the city Radius? It's currenlty at a maximum of 3, I would like the max to increase to 10. There is a mod that did the code but it requires DLL changes.
Larger City Radius
http://forums.civfanatics.com/showthread.php?t=241976

Culture Sizes
http://forums.civfanatics.com/showthread.php?t=511066

At the very end of thread a guy said he converted it to 3.19. I'm going to ask him for the code.

We could, but the code is not structured in such a way that it could be done with an isolated change, nor will the current code scale very well with further radius increases (i.e. It'll run very slowly unless we totally rework how city workable areas operate internally. I don't really recommend it in conjunction with all the other scaling factors in c2c unless someone is repaired to spend considerable time learning g how it all woken and finding a better way to reimplement it (it would take me a few weeks I think if I devoted myself to it, but that's not a priority that is likely to bubble to the top any time soon for me)
 
I think we could stick to only adding more culture levels for now.
It's XML-only and easy, but would be nice.
 
About the radius. I think, we would also need to add a new functionality of scrolling the city view. Having 20x20 in the view at once would probably not be too readable.
 
Can we edit the element that controls the city Radius? It's currenlty at a maximum of 3, I would like the max to increase to 10. There is a mod that did the code but it requires DLL changes.
Larger City Radius
http://forums.civfanatics.com/showthread.php?t=241976

Culture Sizes
http://forums.civfanatics.com/showthread.php?t=511066

At the very end of thread a guy said he converted it to 3.19. I'm going to ask him for the code.

Yes it is certainly possible. The gameplay implications of doing so would be quite far reaching though, and as Koshling said it isn't an isolated change.
 
About the radius. I think, we would also need to add a new functionality of scrolling the city view. Having 20x20 in the view at once would probably not be too readable.

Yeah in the original code the modder said something about map zooming, it required some C++ changes that are too advanced for me.
 
@MrAzure, I think zooming out is not good solution. If you have such big area in the city view, you will probably hardly see what you have at a plot. The food, hammers and gold icons will need to be really small then. Scrolling could be a solution, but it would rather need more effort.

@Koshling, do you know where is the code responsible for modules loading? There is some problem about TD's combat classes, so I would like to check how it is done and where to put some code he needs. I've tried to trace copyNonDefaults calls, but it takes me to some middle steps and I don't have time to trace everything.
 
@MrAzure, I think zooming out is not good solution. If you have such big area in the city view, you will probably hardly see what you have at a plot. The food, hammers and gold icons will need to be really small then. Scrolling could be a solution, but it would rather need more effort.

@Koshling, do you know where is the code responsible for modules loading? There is some problem about TD's combat classes, so I would like to check how it is done and where to put some code he needs. I've tried to trace copyNonDefaults calls, but it takes me to some middle steps and I don't have time to trace everything.

CvXMLLoadUtility.* and especially CvXMLLoadUtilityModTools.cpp and CvXMLLoadUtilitySetMod.cpp

However, it's not (IMO anyway) very 'obvious' code, so this comes with a 'here be dragons' warning.
 
Back
Top Bottom