The State of the AI

Here, various developers can tell you all about what they're up to, so you can yell at them for being idiots. "... and there was a great rejoicing."
Post Reply
Message
Author
User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

The State of the AI

#1 Post by Dilvish »

I decided to try writing up a description of the current state of the AI and my plans for AI development, partly just to help meet people's curiosity about what the AI is up to, partly to help enable better feedback about the AI by making it more clear what what things it is currently attempting to accomplish and what are its known gaps, partly to share my roadmap to better enable folks to suggest additions or other changes to that, and partly to further document the organization for anyone wishing to get directly involved i AI development. Just simply organizing my thoughts more clearly for writing here will probably also help. This first post will focus on characterizing the current state of the AI, and a second post will focus on AI development plans; I'l be updating both as planning and development proceeds.

The first iteration of both of these posts will start out pretty sketchy, but I'll be working on filling in more detail over time.

Overall Coding Structure of the AI
AI Interface
The FreeOrion c++ code includes a primary generic AI Interface component located in teh FO "AI" folder, which specifies a number of function calls the AI needs to support for starting new games, saving and loading games, and for sending and receiving diplomatic updates and chat messages. It also specifies a set of functions the AI uses to generate Orders passed on to the server to actually effectuate the AI's instructions to ships, fleets & colonies, and to control the Production and Research Queues. The current AI is primarily implemented in Python, and so there is a secondary AI Interface layer which deals with actually loading and initializing the Python AI and which provides the Python-specific interface for those functions in the primary interface layer. There is an additional set of Python interface code, located in the FO "python" folder, which provides C++ <==> Python wrappers which the Python AI can use to get information about the various objects in the FO gamestate, such as Ships, Fleets, Planets, other Empires, the AIs Production and Research Queues, etc. These wrappers currently cover much, but not the full set, of information potentially of interest to the AI; additional wrappers are added as AI development creates the need.

The Python AI Codebase
The actual Python AI is made up of a number of modules located in the FO "default/AI" folder (or as otherwise specified by the combination of --resource-dir and --ai-path), as it is a type of content aspect that can be updated without necessarily needing the main FO code to be recompiled. The primary module in the Python AI is in FreeOrion.py; this is the module directly loaded by the AI Interface code (see below) and which has the direct support for those functions required by the AI Interface. It is this module which receives the NextTurn message and corresponding updated Universe info from the game Server, and which then leads various other Python modules through some steps of evaluating the current gamestate and formulating actions to be taken, which are then packaged up as an OrderSet which is sent to the Server. Once the server has received orders from all the empires it starts the next turn, processes the orders, and then sends out the updated Universe information. The FreeOrion module also instantiates an AIstate Python object, in which the AI saves whatever state information it wishes to be maintained in a persistent fashion; this AIstate object gets serialized and passed to (or received from) the Server during game Save and Load operations. I'll add more detailed information about what's currently tracked in the AIstate object and references to other specific Python modules later.

AI Startup
For each AI, the FO Server launches a new process. The process executes the C++ AI client, for which the server has provided some command line parameters. The first two arguments are playername and max AI aggression, followed by the optionsDB parameters --resource-dir, --log-level, --binary-serialization, and --ai-path (relative to the resource-dir, defaulting to "AI"). Also, if the --ai-config option is not empty it is passed as well. The top tier of the client is specified by client/AI/camain.cpp, which pretty much just processes the command line and creates and runs an AIClientApp object, specified in client/AI/AIClientApp.cpp. This in turn creates the primary IA object m_ai, currently a PythonAI as specified in AI/PythonAI.cpp, it initializes the (c++) PythonAI and then waits for messages from the server, which each generally specify some corresponding c++ function of the PythonAI object. When initialized, the C++ PythonAI client immediately initializes a python interpreter, imports a logger and starts up logging, and adds resource_dir/AI (or as specified by --ai-path) to the python sys.path. The C++ PythonAI then uses some boost code to cause python to import the FreeOrionAI module (triggering config setup, below) and to keep a C++ reference to it in the c++ "s_ai_module" variable. The PythonAI client then immediately runs the FreeOrionAI.initFreeOrionAI() function (which currently simply logs that the AI is initialized).

AI config setup When the FreeOrionAI module is imported by the client, it in turn imports freeorion_tools.Timer, in turn importing freeorion_debug.option_tools, which establishes config parameters using a SafeConfigParser object from the ConfigParser module. The config parser may read one or more config files within the ai-path. An initial set of default configs are established by reading "default_config.ini" if present, or otherwise from preset values. Then, if the ai-config parameter is not empty and names a file present in the ai-path, then the config parser attempts to read that file as a config file. Currently the configs simply control aspects of AI timing and associated logging, but it is expected that many operational characteristics will be controlled by it and that there will be a mechanism for different AI empires to get potentially different parameters, to assist with automated assessment of AI charactersitcis. The command line options --auto-advance-n-turns <n>, --auto-quit, and --load <save_game_file> may also be helpful for AI testing.

Continuing OperationAs the game proceeds the server sends various messages to the C++ AI client which generally trigger some function of FreeOrionAI to be looked up and executed. The most important part of this cycle for the AI is that each turn a TURN_UPDATE message is sent to the AI, which includes a new copy of the universe which entirely replaces/supersedes the previous copy of the universe held by the client, this also triggers execution of the "generateOrders" function of the s_ai_module, and then sends a message back to the Server with the resulting Orders. It is important to note that the entire universe, and every object in it, is replaced by this update. There is a plan to move to an incremental update process, but currently the universe is entirely superseded each turn, and the real difference with an incremental update will just be efficiency-- the only way the client can reliably and coherently track objects from one turn to the next is by ObjectID.

AI Analysis and Actions
This section initially covers mostly the basics, I'll be adding further details over time.
  • The AI starts each turn by reviewing the status and ratings of visible fleets of its own and of enemies or monsters; this includes tracking where it may have lost a fleet in battle to a stealthed enemy and then estimating that enemy's strength (though this step may no longer be necessary with current stealth/combat rules).
  • Next the AI updates its exploration info with respect to visible & explored systems.
  • Next the AI reviews its colonies, where it has species capable of colonizing and/or building ships, their populations and their metabolisms. As part of this phase it also characerizes surrounding systems was being within current supply range, or within the range of what would get connected for supply purposes based on current supply tech. Also as part of this phase it rates the observable unpopulated planets based on population, projected production, etc, iterating over each colonizable species currently within the empire & saving info on the best one for each planet. The AI also does a review at this point to check in which systems it has the ability to build outpost bases rather than needing to send in an outpost ship from elsewhere. Planets in those base-able systems are excluded from the regular outpost ranking list, and a separate scoring is done for them, any with a high enough score have an outpost base placed on the production queued for them. Also, for colony rankings, it is no longer just the best score saved, but scores for each colonizing species available to the AI are saved, and during the colony fleet dispatching stage, a colony ship with non-optimal species could be sent if its score is close enough to the optimal one.
  • The AI then performs a similar rating for invadable planets
  • Next the AI assigns available (i.e. currently untasked) scout fleets, colony fleets, and invasion fleets to selected targets. Note above re colony ships.
  • After that comes one of the most critical stages -- assigning military fleets. This is done in a few stages. The AI has a new fleet rating system that takes into account the new shield DR dynamics and which also takes into account the enemy ship designs it is facing. Enemy fleets are assessed to determine colony defense needs, the scouting, colony and invasion targets are assessed to see if military force is needed to go through first, and enemy fleets that are otherwise just within the AI empires general region of space are potentially targeted. After needs are assessed, a round of assignment is done to try giving each task the minimum assessed force needed, then a second round of allocation is done to assign out any further military fleets, to those same tasks, so that they might get the benefit of superior military force. These assignments are recorded as 'missions' for each of the fleets.
  • Next, for each fleet an appropriate set of current orders is determined, orders which will be submitted to the server. These include move orders, colonization orders, etc. As part of assessing potential move orders, the AI considers nearby enemy fleets and whether it is reasonably safe for the subject fleet to proceed, or if it should wait for military backup.
  • After that, any research orders are generated. An initial set of fairly exhaustive research orders is generated on the first turn, but each turn thereafter the AI assesses various conditions to decide if research priorities will be adjusted.
  • Next, Production is evaluated. This has two main components, buildings and ships. Buildings are first evaluated, to see if they are available to the empire and if the designated circumstances are present to trigger their construction. Buildings that do not serve to increase total production, such as various shipyard facilities, are subject to a cap on how much of the empire's per-turn PP can be assigned to them in total. Some types of building will have their construction be accelerated by being moved to the front of the Production Queue. In regards to ships, on the first turn an exhaustive set of ship Designs is created, ranked within each ship class (colonization, outposting, exploration, invasion, military/marks, etc). Each turn thereafter, the AI iterates through its ResourceGroups with available PP (most commonly there will be one group, unless there are supply blocks). The AI determines which Designs are currently unlocked, and of those, which is the highest ranking within each class for which the AI has suitable construction facilities available within this ResourceGroup. Based on previously determined priorities for Colony fleets, military fleets, etc., the remaining available PP will be stochastically assigned to the various ship priorities.
  • Finally, resource production on the AIs various colonies is assessed, and planetary foci are set for Industry, Research, or Growth.
For anyone wishing to work on the AI, the following threads have helpful information:
options to facilitate testing
interactive python console via chat
autocomplete in IDE for python AI Interface


**edit** recent changes are emphasized in bold; latest changes are regarding AI startup and config
Last edited by Dilvish on Wed Feb 18, 2015 3:26 pm, edited 6 times in total.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: The State of the AI

#2 Post by Dilvish »

Here is a first draft, or at least a start of a first draft, my my current plans for AI development. This is not exhaustive, or at least certainly not yet. I'll try to list things in general order in which I hope to address them; this is not quite equivalent to importance, in that along with significance it includes a balancing out of projected difficulty and/or how much of a solution I've been able to figure out so far.

Anyone wanting to discuss some aspects of this should probably post in the Design forums rather than reply here.

Production Queue Management
The AI has some ability to deal with available PP being broken into multiple Resource Groups (in event of supply blocks). I will be continuing to look at refining its handling of multiple Resource Groups. Add support for Space Elevators.

Military Response to Supply Disruption
Partially related to the above, I plan to make the military assignment process pay attention to supply disruption and put a high priority on clearing such.

Outpost bases & colony bases
The AI recently gained the ability to deal with outpost bases, which generally have less delay and risk than building and sending an outpost ship from elsewhere. Next up in this area is modifying the colonization process to include colony bases.

Multiple Ship Hull Tech Lines
The AI will pursue, over time, multiple ship hull techs, accelerating certain ones when favorable circumstances arise. I will continue refining this and the corresponding ship designs. I will look at having the AI defer certain hull techs when systems/planets enabling more favorable hull lines are available; in such circumstances I may have maniacal AIs totally abandon the less preferred techs.

Scouting
follow up on these ideas: viewtopic.php?p=71164#p71164

Military Fleet Ratings
This is an area of ongoing attention.

Invasions
This is an area of ongoing attention.

Military Strategy
There are a number of aspects of military strategy, such as planning chokepoints, disrupting an enemy's supply network, and strategic retreat, that the AI essentially does not do / deal with at all right now (except sometimes through lucky happenstance, and now some capability for repair-retreats). These have varying levels of difficulty. I don't have immediate plans for these, but they are definitely part of the long term plan. I'll be looking at further restrictions to how the AIs merge military fleets.

Alliances
We've had some discussion of Alliances in the Design Forums, largely driven by the AIs but in which a player could sometimes be a part. One of these days I'll start to experiment with implementing the basics.

Misc Todo's & Bugfixes
* make AI aware of Molecular Clouds & their shield effects

I'll be adding more detail & more items here as they come to mind. As a reminder, if you wish to discuss any of this please do so in the Design Forum.
Last edited by Dilvish on Sat Dec 13, 2014 8:33 pm, edited 8 times in total.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: The State of the AI

#3 Post by Dilvish »

Updates, Dec 2014, regarding AI startup and config process, and related command line options.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

Post Reply