AI stops colonising at mid/late game

Describe your experience with the latest version of FreeOrion to help us improve it.

Moderator: Oberlus

Forum rules
Always mention the exact version of FreeOrion you are testing.

When reporting an issue regarding the AI, if possible provide the relevant AI log file and a save game file that demonstrates the issue.
Message
Author
User avatar
Oberlus
Cosmic Dragon
Posts: 5715
Joined: Mon Apr 10, 2017 4:25 pm

AI stops colonising at mid/late game

#1 Post by Oberlus »

Playing 2017-06-06.4833950_Test_Win32 (however I have noticed in previous versions what I'm telling in this post).

In most (if not all) my games with around 30 or more systems per player, I've noticed that AIs (all of them) stop colonising available planets from a certain point (usualy around turn 200). With available I mean that those planets are in systems with already one or more colonies or right in between their area of control (and supply).
This behaviour cripples the AI performance for late game, making games boring from a certain point even if you started in disadvantage.

My previous game was 1vs1 (AI turned out to be a gysache, I was human, so it was going to be an interesting game), 100 systems 2-arm spire. I started near the tip of one of the arms while the AI started close to the core. I also was fenced in between some sentinel systems and had three kraken nests behind those sentinels (one on the very end of my spire arm and two next to the galaxy core) that swarmed my spire arm, slowing my research and colonies production down for a while.
Until turn 200 or so AI had more colonies and considerably more PPs than me (around 50% more than me, while RPs were on par), but from that turn on it stopped getting any new colony, despite having maybe as much free planets at it already had. I'm certain that the reason was not the habitability of the planets, because AI already had three or four native species (including humans since it stole one of my planets around turn 150 and I couldn't get it back for a while) as well as most growth techs. I'm also sure it had enough PPs to devote to colonising while still keep a fair military production to at least keep me at bay, since I had less PPs and I was doing right that: colonising while fighting back.
We had been battling each other at the junction between my spire arm and the galaxy core since turn 150 until turn 250. At the beginning I was just withstanding the blows with defense focus, comsats and all the military strength I could gather. From turn 200 or so I began to be on par with it military-wise thanks to better stealth carriers that allowed me to wreck havoc on its fleets with little loses on my side. From then I had same number of colonies than it and my PPs began to surpass its. The number of colonies (and planets) graphs showed that the AI had stopped getting new planets (flat line). From turn 250 I was already taking planets from AI in the galaxy's core. I had already scouted entire galaxy and saw no less than 20 uncolonised systems with nice planets well into AIs territory, as well as available planets within systems with one or more colonies.

Is the AI team aware of this?

If the answer is no, I will save and upload some logs for complete games (though this will take me some time, I'm having little time to play).
Regarding this, I've made a suggestion about allowing saved games to keep the logs of the whole game, to help keep track of all the AIs' decisions.

Morlic
AI Contributor
Posts: 296
Joined: Tue Feb 17, 2015 11:54 am

Re: AI stops colonising at mid/late game

#2 Post by Morlic »

Oberlus wrote: Is the AI team aware of this?
Kind of. As in: I noticed the behaviour but did not prioritize it enough to really look into it.

If you have games that showcase that behaviour, then yes, please post the savegames (preferably around the point where the behaviour started) and/or logs. I will look through those and see if there is an obvious bug / easy improvement to the AI logic.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: AI stops colonising at mid/late game

#3 Post by Dilvish »

Yes, it's a great testament to the AI changes over the last several months that we can notice this limit now. The intent was

Code: Select all

# significant growth barrier for low aggression, negligible for high aggression
but now it has no longer been negligible for high aggression AIs. I've been experimenting with a change and just went ahead and committed it. It's an improvement, but it could be that we should just actually entirely remove the limit for aggressive and maniacal AI's rather than try to make it "practically-removed". Further feedback welcome.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

Morlic
AI Contributor
Posts: 296
Joined: Tue Feb 17, 2015 11:54 am

Re: AI stops colonising at mid/late game

#4 Post by Morlic »

I'm not sure that is the relevant fix, Dilvish (though it probably does not hurt).

The limit you changed is linear in the number of turns and is used both for colonization (https://github.com/Morlic-fo/freeorion/ ... AI.py#L249, https://github.com/Morlic-fo/freeorion/ ... AI.py#L282) and invasion (https://github.com/Morlic-fo/freeorion/ ... AI.py#L333).

So that should cover all means to acquire new planets (except for having already an excess number of ships which seems rather unlikely). As the limit grows in time, I can't see how that limit would suddenly cause a stop.

If I had to guess, I would expect that the AI has too high priority into military ships at late game. I have often seen that the AI requests unreasonable high amount of defensive resources to a multitude of systems in the later stages. I would expect that this reflects in the priorities so that no more colony ships are build and also that this is the reason why late-game the AI is very rarely attacking.
So, my assumption is this is a side-effect of the becoming significantly bad military allocations in the later stages of the game. I do not think the underlying issue is trivially fixed in that case.

A stop-gap solution could be to have the AI always build some colony ship if it has possible colonization targets and has some certain minimum PP output (probably based on the value of colonies), essentially overriding military emergency situations.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: AI stops colonising at mid/late game

#5 Post by Dilvish »

Morlic wrote:I'm not sure that is the relevant fix,... As the limit grows in time, I can't see how that limit would suddenly cause a stop. If I had to guess, I would expect that the AI has too high priority into military ships at late game.
I did not include in that commit some related additional logging which had revealed to me that this limit was indeed acting as a significant brake on the AI, but I'll plan on cleaning that up a bit more and putting it in to facilitate further honing of such things. In the meantime, trust me, this indeed was in many games imposing a very strong brake on the AI growth, such that a player like Oberlus, just observing the game and not doing a detailed analysis of AI logs, would likely perceive the AI as stopping colonizing rather than nearly-stopping colonizing. And yes, it is my general recollection that this had been affecting invasions as well, though I don't recall if I had really conclusively established that via logging (but I think this change did improve the observed behavior).
A stop-gap solution could be to have the AI always build some colony ship if it has possible colonization targets and has some certain minimum PP output (probably based on the value of colonies), essentially overriding military emergency situations.
You are certainly free to explore other approaches to the problem, but I suggest you give this approach a solid try before you invest time into something else.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: AI stops colonising at mid/late game

#6 Post by Dilvish »

I also just pushed another commit which is a bit related (just preventing integer division in some calcs); I don't recall now if I had really established how much difference this last one makes, but I am confident it doesn't hurt and I am inclined to think it helps.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

Morlic
AI Contributor
Posts: 296
Joined: Tue Feb 17, 2015 11:54 am

Re: AI stops colonising at mid/late game

#7 Post by Morlic »

Dilvish wrote:
Morlic wrote:I'm not sure that is the relevant fix,... As the limit grows in time, I can't see how that limit would suddenly cause a stop. If I had to guess, I would expect that the AI has too high priority into military ships at late game.
I did not include in that commit some related additional logging which had revealed to me that this limit was indeed acting as a significant brake on the AI, but I'll plan on cleaning that up a bit more and putting it in to facilitate further honing of such things. In the meantime, trust me, this indeed was in many games imposing a very strong brake on the AI growth, such that a player like Oberlus, just observing the game and not doing a detailed analysis of AI logs, would likely perceive the AI as stopping colonizing rather than nearly-stopping colonizing.
I perfectly believe you that this limit will slow down an AI that has a good game and that increasing it is a very good idea.
I am arguing that it should not lead to a complete halt of the AI expansion unless there is another logic error / bug somewhere in regard to this.

Which aggression level did you play on Oberlus?
For typical aggression (=3), the limit is increased by 1 roughly every 4 turns given the old formula. I can not see how that can possibly cause a "flat line". For aggression level 2, we have an update rate of 1 per 8 turns. That still seems way to steep to cause a complete halt.

Am I overlooking something simple or some other relevant part of the code?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Oberlus
Cosmic Dragon
Posts: 5715
Joined: Mon Apr 10, 2017 4:25 pm

Re: AI stops colonising at mid/late game

#8 Post by Oberlus »

Morlic wrote:Which aggression level did you play on Oberlus?
Usually Maniacal, sometimes Aggressive. I'm not sure which one I was in the case I describe in the opening post.
I just checked I have a saved game for turn 117 of that game, although the colonisation halt happened way after that (maybe 200 or 250). I no longer have the autosaves for that game since I started another game last night. I will replay it until AI shows the halt (and save the complete logs before next FO session) for you to tinker with, but not sure when, it may take some days (got a pretty damn deadline this week).
Morlic wrote:If I had to guess, I would expect that the AI has too high priority into military ships at late game. I have often seen that the AI requests unreasonable high amount of defensive resources to a multitude of systems in the later stages. I would expect that this reflects in the priorities so that no more colony ships are build and also that this is the reason why late-game the AI is very rarely attacking.
All this seems very plausible. I had also noticed that AIs cut down attack aggressivity on late game, although I blame that on my fleets getting bigger than theirs. However, I had also seen that the losing AIs are never fully conquered by their stronger neighbours if they made it into late game despite having nearly nothing to defend their last few (and usually nice) planets.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: AI stops colonising at mid/late game

#9 Post by Dilvish »

Morlic wrote:I am arguing that it should not lead to a complete halt of the AI expansion unless there is another logic error / bug somewhere in regard to this...For typical aggression (=3), the limit is increased by 1 roughly every 4 turns given the old formula. I can not see how that can possibly cause a "flat line". For aggression level 2, we have an update rate of 1 per 8 turns. That still seems way to steep to cause a complete halt.
It's still not clear to me that Oberlus was necessarily talking about an actual complete flatline/halt instead of perhaps simply referring to a drastic shift from exponential growth to such slow linear growth that it appeared to be relatively flatlined.

Oberlus, please try things out with current master and let us know if you are seeing the same issues still or not, and if they are at least delayed until later in the game (I think that the changes I made should at least substantially delay the issue, but some further changes to the same bit of code would likely be needed to totally prevent the possibility of the problem in long games, especially with high planet density).
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Oberlus
Cosmic Dragon
Posts: 5715
Joined: Mon Apr 10, 2017 4:25 pm

Re: AI stops colonising at mid/late game

#10 Post by Oberlus »

Dilvish wrote:It's still not clear to me that Oberlus was necessarily talking about an actual complete flatline/halt instead of perhaps simply referring to a drastic shift from exponential growth to such slow linear growth that it appeared to be relatively flatlined.
It was completely flat, i.e., straight and horizontal line for a sustained number of turns. Then it started going down once I began taking planets from AI. Prior to that, the growth was hardly seen as exponential (it was, but it could be approximated with a linear function with little error, specially for the two last thirds of game).
Sorry for not attaching a screenshot (I hadn't access to FO when posting, I'll do once I replay the game).
Dilvish wrote:Oberlus, please try things out with current master
Sure. I'll report results. Just to make me sure, "current master" is the latest test build that Vezzra publishes, right?

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: AI stops colonising at mid/late game

#11 Post by Dilvish »

Oberlus wrote:
Dilvish wrote:Oberlus, please try things out with current master
Sure. I'll report results. Just to make me sure, "current master" is the latest test build that Vezzra publishes, right?
Ah, the weekly test build does correspond to current master at the particular time when it comes out, but the commits I was referring to are not in last week's build. They will be in the next build that Vezzra puts out.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Oberlus
Cosmic Dragon
Posts: 5715
Joined: Mon Apr 10, 2017 4:25 pm

Re: AI stops colonising at mid/late game

#12 Post by Oberlus »

Dilvish wrote:It's still not clear to me that Oberlus was necessarily talking about an actual complete flatline/halt instead of perhaps simply referring to a drastic shift from exponential growth to such slow linear growth that it appeared to be relatively flatlined.
Last game I'm playing is similar to the one I described in this thread: 1vs1 maniacal, 100 systems 2-arm spiral. Currenty I'm turn 200. Very similar development. Once AI and me began our quarrel next to the junction between my arm and the core of the galaxy, AI has drastically slowed down (not completely halted) colonisation speed. Maybe Dilvish's change is to be blame of this improvement.
I shall finish the game before any conclussion, but so far I can tell AI's colonisation speed is still too slow. AI's colonies' growth was linear until we began smacking each other, and then it has become extremely slow but still linear (around 5 times slower than before), in spite of having many available planets to colonise well into its territory.

Should I see faster AI's colonisation if I played typical AI instead of aggressive/maniacal?

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: AI stops colonising at mid/late game

#13 Post by MatGB »

Bear in mind: the game is balanced and the AI designed for far more empires, 1:50 is outside the recommended balance and the AI will start doing stranger things at that point, if fighting it will put resources to warships, perhaps excessively so because it can't "know" that it's just you and it or the galaxy size.

I don't know the details of how the AI sets priorities, but I do know that once in a fight the higher priority is that fight, and you're at a fairly edge case setup in terms of balance there.
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

User avatar
Oberlus
Cosmic Dragon
Posts: 5715
Joined: Mon Apr 10, 2017 4:25 pm

Re: AI stops colonising at mid/late game

#14 Post by Oberlus »

Yeah, it makes sense that on more populated spaces (<20 systems per player) AI gives all priority to warships once combat begins, since at that point chances are there are few planets for colonisation. However, I can beat them easy since AI neglects colonisation (i.e. cripples its own production/research potential) and that gives me plenty of space to surpass its PP/RPs production since mid game (100 turn or so).
I'd say that colonisation/militarisarion balance should depend on the perceived menace from encountered enemies as well as the colonisation opportunities. If you let your enemy conquer one of your planets to get two new colonies on another place that pays you back and allows you to recover the lost planet as well as conquering a few more (and then it's over).

Side note, I find contradictory that you and some others told me AI behaves better (harder to beat) when playing on galaxies with 40+ systems per player, and now this comment seems to state the opposite :)

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: AI stops colonising at mid/late game

#15 Post by MatGB »

If I actually said that, either I was wrong or I've changed my mind ;-)

I can recall Dilvish saying that's how he prefers to play and I tend to prefer sparser maps, but I find the AI most challenging on High planets and in situations where they can start conquering each other relatively early: sometimes they fight to stalemate but normally at least one does well enough that it's actually a challenge by the time I find them.

Sparse maps are interesting, but the AI isn't designed for it although it's been improved in that direction. But, specifically when I'm testing effects or changes to features like, for example, engine speed of the species mix, I try to play within the recommended specs. High planets definitely makes them more challenging, it makes it a lot easier to expand in the early game and it's that that can sometimes give human players an advantage because we can think steps through better if there are fewer worlds to grab.

If you want a real challenge at the moment play a 200ish system map on High Planets with 10+ AIs: and don't play Egassem (their nerf needs to come in soon they're really OTT in that sort of setting).
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Post Reply