1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

Ai

Discussion in 'Civ4 - Community Enchancement Project' started by Olleus, Jun 2, 2006.

  1. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    I think just one global switch for the new AI should do it. We shouldn't have to have swithes to enable/disable every part of it.
     
  2. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    Personaly I dont see a need for that, people universaly want strong AI's. Theirs are also going to be lots of AI changes that just involve it understanding/using a new feature and having these turned off could cause problems when the new feature is used.
     
  3. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    I think such a global switch is recommended. I want strong AIs as well, but in case it gets to strong I would like to make it weak somehow.

    12m
     
  4. SpoiledFruit

    SpoiledFruit Chieftain

    Joined:
    Mar 11, 2006
    Messages:
    72
    Location:
    Boise, Idaho
    Why not just add a dropdown menu for an AI option instead of just a checkbox. That way we can have easy, medium, hard, and impossible settings in one switch. When an option is selected it sets a different set of algorithms that the AI runs from.

    The new features could be just a simple (!ModElement) so if the python elements are not there, the AI does not take that section of code into account, thus reducing the chance for errors and improving modability.
     
  5. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    If the AI were made dramaticaly smarter wouldn't you just drop down a difficulty level? The difficulty levels all work by giving the AI artificialy bonus || penenlty under all of them the AI "thinks" in adsactly the same way (dumb).

    Now I know their was some though about alowing AIs to be instantiated and played against each other to compare their effectivness. That seems a better strategy so you could play a regular Firaxian AI against a supped up AI and see how decisivly or consistently you could win.
     
  6. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    It definately doens't need a drop-down. Either on or off should do.

    I would suggest having the on/off switch as there may be some mods for which our the deafault AI may be better - especially if we optimise our AI to default game... though I supose if we do it right there shouldn't be.
     
  7. chrusion

    chrusion Chieftain

    Joined:
    May 5, 2006
    Messages:
    36
    Location:
    Seattle, WA
    One switch sounds good for general use.
    But I think AI modules should be individually switchable from Python.
    This is only from the "We are making a mod for modders" and the "AI is a pain in the south end of a north bound unit to do and test properly" points of view.
    Of course, defining modules, dependancies, and so forth may be a nightmare of its own. So take this with a grain of salt.
     
  8. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    Sorry for being so inactive in the here, having a huge project at school and exams as well. However when it is finished (around next week), I will be able to contribute more.

    About genetic algorithms is an interesting approach. Having a huge interest in artificial life my self and a bit experience with this. However it might be a bit overkill to have an exact GA. Maybe we should drive toward some dynamic programming instead, or EP, maybe GP, or even a neural network :crazyeye: however it is not important what we end up with, as long as the AI becomes 500½ % better at decision making and playing the game, so you, as a human player, would get an equal match, without the AI cheating by getting unfair help (practicallymaking multiplayer obsolete;)).

    Back to main point:the AI needs major rework on all areas. And I think could be a way to go is a top-down approach on this.

    (OBS.: he is generic for both he/she/it ;))

    I mean that the AI need to make high level (long term) decisions, and going down the levels/layers, down to simple direct decision like unit movement and city production/specialist etc. For example the AI decides that (Overall goal) "I should dominate this continent by 1500 AD", then at the immediate goal layer it decides "I will build up my army so I can attack Ceasar", then at city level it should ask the question "What should I produce in this city if, I would like to fulfill my immediate goal, giving the situation my: empire is in and the city itself is in." etc. etc. down to unit level.

    The Overall goal and immediate goal should not be revised, unless something drastic happens, like the finding of a new uninhabited continent (maybe overall goal here) or someone is near completion of the spaceship (both overall and immediate goal), or declaration of war (immediate goal changes). We can make as many layers as we think is needed. The low levels, should be revised, not necessarily changed, but evaluated again, each turn or every second turn. Very lowlevel may need to be evaluted more than once, each turn (as stated in above posts).

    General:
    The higher the decision level is, the more abstract it is in terms of actions, and is more influenced by the leader. And vice versa. The lower the level, the more concrete it is, in terms of actions, meaning it can be directly translated into an action in the game (move unit from x to y for example), while this is more influenced by higher levels, other decisions, the world around it etc.

    It is important with the top-down approach, that information, decisions and factors propagate up and down in the system. So another players city placement (losing a valuable resource for instance), would propagate up to the war decision layer(and thereby the immediate goal layer or something), and instead of a purely random factor decides war or peace, it should incorporate much more factors, so it would seem both more realistic, but also unpredictable, so exploits would be rare or extremely hard for the player to take advantage of (applies to all layers).

    A comment on war:
    Instead of just having the AI going toward the nearest city in war, it could do several things instead, based on a lot of factors. Like: if the AI determines that a defensive war is better (because conquering cities, having better defensive units, technology level etc.) then it would not attack, may at the most probing with a few units. Based on leader personality, he could move all his offensive units through the land and pillage all he can, or he could go against the nearest city, or split his army in two, attacking two different targets, and/or use sea transporters to sail around and attack from the rear (and not only 2 units...), eventually the AI could scout with spies, missionaries etc. before declaring war and determine his attack plan. And I could go on and on.

    In addition the AI needs to know when he is heavily losing. Right now only success is record for the player and the AI, so having a success of 0, doesn’t really meaning that it goes decent, and likewise with a success of 50 or 100, if the AI have lost all but one city, then he still thinks has some kind of success.
    (Check AI_endWarVal in CvTeamAI)


    I know this post is a bit ...Random?? and hard to understand, however I am very busy and only wanted to comment quick.
    I will try to come up with a more concrete system later. Please comment if you understand my post :D

    (Edited for clarification, spelling mistakes and random babble)
     
  9. Olleus

    Olleus Warlord

    Joined:
    Oct 30, 2005
    Messages:
    6,478
    Location:
    England
    I agree with you completely NikG. To make the AI better, we need to make it more 'human'.

    All of the best civ players are human (obviously) so the AI needs to think more like a human. I know that this is impossible as AI's can think in the true sence of the word, but we can get close to this by having it mimic the style of play of a human.

    The best way to do this is to play a game very carefuly and write down every descison that you make and why you made it. For example, "Moved a longbowman from Paris to Orlean as I think that it is in danger". You can then go in even more in detail and say "Orlean is in danger because war with China is iminent". Then you can say "I think that because they have an SoD on our border.

    I know that this would take ages but it would help us understand how the human player thinks. By knowing this, we can rebuild the AI to copy it.
     
  10. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    Glad you like my ideas Lord Olleus.

    And I think that it would be absolute amazing if someone would do the experiment you are proposing (which I really like). Actually it could just be a small map with 3 or 4 civs, with the quick option. The posting in, we could analyze this together.
     
  11. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    It'd probably be best to get a player from the Stratagy forums where they are all good. One slight trouble would be that the humans would probably be playing against the AI - and so some of their actoins might be justified by predicing AI behaviour... which obviously doesn't help if we're changing that behaviour.

    Sounds like quite a boring job.

    About genetics - you've kinda lost me with EP and GP. What are they? I just did a quick poke through wikipedia and couldn't find anything that they might stand for.
     
  12. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    EP is evolutionary programming and GP is genetic programming. We even have EA, evolutionary algorithm. Computer scientist just love acronyms... :)

    But about the boring job. Well it might not be so boring. Actually it would just be like written one of those stories, just more precise and detailed... :crazyeye:

    But I don't think it matter if it comes down to preceding AI behavior. I mean what a human determines to do of actions is based on the AI's actions, not how the AI decides the action, so in the end it really doesn't matter. Of course when the AI changes, the AI would make more complex analyze of the situation at hand, and then make better decisions, which in the end corresponds to better actions, if I might say so. In the end making the AI a better player. So a human may be more careful with a new AI, compared to the old one. And as such the human doesnt really alter the decisions of his own because the actions of the AI gets better.

    Btw if we get a good AI working, we could end up with a AI that doesnt need 75.000 units. And thus making it even better in the tech race and other areas as well.
     
  13. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    Slightly confused.

    What is the difference between a GA and GP. Seems a GA is GP isn't it. EP is like GP, but doesn't have crossovers, right?

    Why would an Evolutionary algorithm be better than a Genetic one? From what I can gather it is similar, but doens't involve crossovers, which seem to me to be good things.
     
  14. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    Most of the field in artificial life is largely somewhat interchangeable.
    Originally they had distinct meanings and uses, however with all the progress in the field, it became similar, and the even the real difference is hard to tell (actually I cannot categorize one precisely).
    Most often though the following apply:
    (anyone with a degree in Artificial Life should correct me if I am wrong, until I get my own degree ;))
    Trying to avoid technical stuff, all uses genetic operators in varying degree. How the next generation is chosen, which to use compared to the fitness landscape etc.
    - EA, is often used as a generic term of all different kinds of algorithms and more, that uses evolutionary processes.
    - GA, use most often a static highly specialized fitness function, with a static representation, and fixed length (not always). The population is often also static and there is no environment as such, and therefore no interactions between the agents. Mostly used for optimization. The representation are often highly abstract.
    - GP, are in theory computer programs themselves. Often written in a pseudo implementation specific programming language (but not always), running in a virtual computer, which is just a piece of software mimicking a computer. (Or it could be said it is a kind of interpreter instead). The fitness function are still specialized, while the representation is concrete, and it is variable in length. Most often used with the LISP programming language.
    - EP, much like GP and GA, but without getting to technical it would be a waste of time to describe the real differences.

    And there is much more.
    But there is another kind, which all out from the normal category. Namely software like Tierra and Avida. Here the fitness function are more abstract, and living entities co-evolve with the environment around them. The representation is like computer programs (hence it can be said it is a specialized form of GP). There is even one AL software without any form of fitness function. DarwinBots (which I am my self a member of, this is an open source program, which is developed by people who love AL). The fitness function is not in anyway coded in the program, and the function is only true darwinan selection, ie. survival of the fittest.

    For more information check the net, and wikipedia isn’t the best source, there are many great sites out there…

    Links:
    Tierra (AL software)
    Avida (AL software)
    DarwinBots (AL software, large community)
    GreyThumb Blog (Nice blog about Artificial Life)
    EvoWeb (Not updated :sad: )
     
  15. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    A side note:
    It is hard to find very good sites describing AL in detail. The best would be to buy some books (Amazon have plenty), and/or program some youself.

    My knowledge about the field, mostly come from books I have bought and by programming a wide range myself (however only at hobby level, so nothing serious and therefore only for my own amusement).
    I study computer science (or I hate that word) at the university of Copenhagen, and intenting to take a degree in AL. However there is no courses in AL in my country(at all, Denmark only have 5 mill. inhabitants), so either I just (re-)invent the field myself or go study abroad. ;)
     
  16. Olleus

    Olleus Warlord

    Joined:
    Oct 30, 2005
    Messages:
    6,478
    Location:
    England
    Is this actualy feasible for a program as complex as Civ4?
    Having a fully genetic AI would require millions of generations before we see a significant improvement compared to what we have now. It would also be a phenomonal amount of work as we would have to write a program which says which AI did better in a particular zone (combat, city location, ect...) to take the best bit of each AI and combine them, if we didn't do this then our final AI could be very good in one feel and awefull in another.

    The only way we could do this is to create very specific senarios and have the AI play against each other in it. I am talking about something like they both start with x warriors, y archers and z axemen. This way we could perfetionate the AI bit by bit, and then combine it manualy to create a super-AI. This way should make it faster and easier to test, and it would allow us to have AI vs Human games, which is the ultimate test.
     
  17. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I think this could have some potential as a method to make AI's smarter but it requires a good deal of foundational work before it could even begin, fortunatly we already had plans for a lot of that work and it would be fruitfull even without the extra Genetic stuff.

    We would need a means of creating an AI instance so each AI player in the game would actualy do things differently. This goes beyond the parameters on the Leaderhead which from what I understand just effect the AI's diplomacy not their internal managment or warfighting tactics. We would need to expose as much as possible the parameters outside the game preferably in XML, it might even be possible to get some kind of genetic Nodebased logic encoded their. Ofcorse their would need to be a lot of compartmentalization as Olleus points out, a little sub-AI to evaluate a particular desision that needs to be made.

    This is all VERY VERY ambitous at this point we should look to doing things that are within our imediate grasp or are a slight stretch (I find that gives me imediate gratification and builds skill to extend my reach without getting bogged down in a project you never finish).
     
  18. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    Yes, I think it probably is. Either it could be oursourced to many computers, or it could be run for several months on one... this is assuming that Civ doesn't crash while you're doing it of course. The amount of generations needed would depend on the generation size.

    Determining the fitness function is definately the hardest part. If this were to be done I would suggest basing it mainly on the outcome of the game rather than individual actions, and then averaging the result over 10 or so games.

    I think a full 10 games would take about 20 minutes per AI (remember we are running many at once), so generation sizes of 40 would take about half a day per generation. The question really is, how many such generations would you need for meaningful results? I don't really know, but I want to have a go at finding out!

    It's not even that large of a project, is it? The basic mutation & combination algorithm should be possible to write in a few hours, and then you just have to track down all the variables you want to change, feed them in, and do a little fiddling to get the computer to repeatedly play games against itself.
     
  19. NikG

    NikG SDK Lover

    Joined:
    Nov 30, 2005
    Messages:
    170
    Location:
    Glorious state of Denmark
    Well okay, I may have started something that I shouldn’t have done... :blush:

    I can't quite follow you here TGA, do you mind explain it more detail?

    (I refer to genome as an AI that needs to be evaluated/tested)

    1 game with 18 different genomes(or AI players) takes how long?

    We assume we have 1 computer, 1440 minutes a day ;)

    In each game we could test 18 different genomes, if a game takes (guess?) 20 minutes, we could have 72 games a day, which equals to 1296 different genomes, however the important stuff is not to evaluate as many as possible, but to evolve successful strategies, so they actually become better. If we say each game is a generation, we could have 72 generations each day, so roughly we need around 20 weeks to get a generation count of 10.000, which is very low (possible we should aim for 50.000). 10.000 generations equals 720.000 genomes evaluations, which is modest. But remember many genetic algorithms have 1000s (if not more) in population size, running for 10.000s generation (which have a really concrete representation compared to what we are going to have, and still evolution is slow, on some super computers they even have population sizes in the 100.000s).

    However all this is only of secondary importance. The fitness function and the representation is a very important aspect of the model we are trying to construct.
    A fitness function of the outcome of the game is easy, but very narrow, and there are so many factors that influence a game, but still it will roughly split the genomes that can't even do some good at all (like not settling at all, or not researching, not building any units etc.), but it could be discussed.

    The representation is very important. TGA you are talking about having variables, which is just like optimizing a differential equation. But this is much more than that. This isn't optimizing, the AI have algorithms to determine what to do, yes based on numbers, but the AI is more than the sum of its parts ;). It is the algorithms we need to represent.

    So without sounding too discouraging, this is just a huge undertaking. No one it the field of AL has every tried such an ambitious project, as this is.

    Sorry.
     
  20. SpoiledFruit

    SpoiledFruit Chieftain

    Joined:
    Mar 11, 2006
    Messages:
    72
    Location:
    Boise, Idaho
    Why do I get the feeling the AI will be the biggest, hardest, most time consuming part of this upgrade. If we are going to do this right, we will need a large group of programmers dedicated to the AI alone to get it out in a reasonable amount of time.

    Somehow I do not think SourceForge is going to handel this type of development to well. This portion is going to change VERY often and will require a much more powerful and user friendly interface.
     

Share This Page