Problems with game speed

Problems and solutions for installing or running FreeOrion, including discussion of bugs if needed before posting a bug report on GitHub. For problems building from source, post in Compile.

Moderator: Oberlus

Message
Author
JoltMe
Space Krill
Posts: 3
Joined: Fri Apr 29, 2016 7:06 am

Problems with game speed

#1 Post by JoltMe »

I cant figure it. I was using a laptop with a quad core AMD cpu (2.8ghz) and 8gb ram. When I would advance in the game to say 350 turns, it would make me wait like 5 minutes for a turn...i upgraded to a quad core xeon processor (3.8ghz) with 20gb ram and i can get to around 500 or 600 turns then the waiting begins. What do i need to play this game? It is like the funnest game ive played in a long time, but if i cant even finish a game because it will take me a couple hours to get to where it starts slowing down, then a week to finish after that...im 50 now...i dont want my next birthday to pass before i win one SMH...

AndrewW
Juggernaut
Posts: 791
Joined: Mon Feb 04, 2013 10:15 pm

Re: Problems with game speed

#2 Post by AndrewW »

Can try turning off some of the effects and setting the frame rate limit. It can start to bog down later on. Smaller galaxy may also help. If you can reduce the number of ships this can speed things up (ie: destroy the big AI fleets, build fewer bigger ships instead of hoards of smaller vessels).

User avatar
Vezzra
Release Manager, Design
Posts: 6095
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Problems with game speed

#3 Post by Vezzra »

What is your game setup (number of systems, number of AIs, etc.)? Because usually a game shouldn't last longer than 200-300 turns (depending on game settings), maybe a bit more, but not 500+ turns. Unless you play really huge maps...

That said, FO doesn't make much use of multiple cores, and doesn't need that much RAM (8GB should definitely be enough, so having more than that won't help). Single core performance is what matters most.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Problems with game speed

#4 Post by Dilvish »

Vezzra wrote:That said, FO doesn't make much use of multiple cores, and doesn't need that much RAM (8GB should definitely be enough, so having more than that won't help). Single core performance is what matters most.
Hmm, I haven't looked into performance for quite a while, but I thought that the AI processes would wind up getting distributed across all the available cores, and so multiple cores would noticeably help-- or do the server or human client wind up being the primary bottleneck (or I suppose that might depend on just how many AIs you have)?

It really seems to me that my recollection in a big game is that the AIs can take an amount of time comparable to the server update stage.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Vezzra
Release Manager, Design
Posts: 6095
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Problems with game speed

#5 Post by Vezzra »

Well, I guess that depends on what actually causes the long turn execution times. Is it the AIs taking so long, or is it the actual server-side turn execution? If the AIs are the bottleneck, then yes, multiple cores should help a lot, as the AI processes should get distributed across the cores.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Problems with game speed

#6 Post by Dilvish »

JoltMe: By biggest suggestion to you, to faciliate big/long games like that, would be to play with Monsters set to Low or None. With Monsters set at Medium or High, it seems kinda easy to me for them to procreate out of control if the AI empire spacing leaves too many open large spaces where the monsters reign. The same kinda of thing can happen a bit just with the AI empires building a ton of ships, but since they tend to group them into fleets it doesn't seem to me to cause quite as much lag as a huge number of roaming monsters does.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Problems with game speed

#7 Post by Geoff the Medio »

Log files from after playing a turn when the game gets very slow would be helpful. Or a save file from when it starts happening. Possibly would help to update to a new test version, if the issue is being seen on an older release.

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: Problems with game speed

#8 Post by MatGB »

JoltMe wrote:I cant figure it. I was using a laptop with a quad core AMD cpu (2.8ghz) and 8gb ram. When I would advance in the game to say 350 turns, it would make me wait like 5 minutes for a turn...i upgraded to a quad core xeon processor (3.8ghz) with 20gb ram and i can get to around 500 or 600 turns then the waiting begins. What do i need to play this game? It is like the funnest game ive played in a long time, but if i cant even finish a game because it will take me a couple hours to get to where it starts slowing down, then a week to finish after that...im 50 now...i dont want my next birthday to pass before i win one SMH...
What version are you playing with.

Also, how big a game are you playing?

As others have said, the game is meant to play for that long, unless you're on a huge map then completion by turn 300 should be the norm. There are multiple things that can cause late game lag, including large numbers of ships per empire and a large number of inhabited planets: the reason the version matters is because we've put in work over the last few cycles to reduce the effect of this lag by improving the efficiency of the scripts and the code.

But the absolute best advice is to play the default setting galaxy size (ie 150 systems) and build up slowly, huge maps when you're not experienced enough to complete quickly will drag down inevitably: I regularly play 400-500 system maps on a machine with far lower specs than you, but I know the game backwards and rarely play past turn 200.
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

JoltMe
Space Krill
Posts: 3
Joined: Fri Apr 29, 2016 7:06 am

Re: Problems with game speed

#9 Post by JoltMe »

I have reduced the frame rate to 15 (from 30)...cancelled all fancy features...gave a tiny boost...not much...once it hits the higher number turns...minutes...i still think it should shell out to ram...everything that isnt in play...oh well...

User avatar
Oberlus
Cosmic Dragon
Posts: 5713
Joined: Mon Apr 10, 2017 4:25 pm

Re: Problems with game speed

#10 Post by Oberlus »

JoltMe wrote:I have reduced the frame rate to 15 (from 30)...cancelled all fancy features...gave a tiny boost...not much...once it hits the higher number turns...minutes...i still think it should shell out to ram...everything that isnt in play...oh well...
Hi. I'm suffering the same issue. I think you should answer the questions rised by Vezzra, Geoff and MatGB:

- What version are you playing with.
- What is your game setup (number of systems, number of AIs, etc.)?
- Log files from after playing a turn when the game gets very slow would be helpful. Or a save file from when it starts happening.



My case:

Machine:
- Windows 10.
- i5 2500K 3.4 GHz, 16 GB RAM.
- FO 0.4.6 (freshly installed a few days ago).

Game set up:
- Spiral 4-arm
- 200 systems.
- Starlane freq.: High
- Planet density: High
- Monsters: Low
- 7 IAs (aggressive).

Experience:
I began that game yesterday and I devoted a quite a time to reach turn 200 (still reading the different techniques/buildings/hulls/etc. and figuring out best strategies), when I noticed the relatively long times of unresponsiveness for turn changes, around 10-15 seconds (not a problem indeed). At turn 250 it was taking more than 60 seconds, plus some lag (seconds wide) began to happen in the IU (between turn-changes), at actions like clicking on a different system in the map or opening the production queue.
Other applications (browser, libreoffice) didn't show any unresponsiveness even when alt-tabbing to them while FO was calculating next turn.
At turn 269 it was already late so I saved, waited to the save window to close and exited (resign + quit) the game.
This morning the game I saved was corrupted (it was in the folder but FO won't show it in the load or save windows). No problem since auto-save had a copy of the previous turn for me. Loaded and noticed that the time to change turns is now under 5 seconds (while yesterday it was 70-80 seconds at the time of saving). Right now I'm on turn 290 and it is still under 10 seconds.

Thoughts:
It seems to me that lag increases over time way faster than the number of colonized planets, number of fleets, frequency of combats, etc.
That and the fact that exiting FO and reloading game back makes it quite smooth again makes me think that there is something odd and wrong, maybe in FO's memory management, and more specifically its data structures. I should have checked memory usage but it didn't seem to be the problem (raw RAM comsuption). I presume it has to do with temporal structures (indexes and the such) that get cluttered over time and cause very slow read operations.
Also, I noticed the AI hardly groups ships into fleets. Most of the time each AI under my radars keeps the bulk of its army in a planet in a myriad of fleets composed of 1-3 ships each. If AI makes calculations for each of its fleets (instead of for each ship regardless of grouping), then I wonder if it would reduce lag to induce AI to merge fleets in same system with same aims or no aim.

Suggestion:
I think the corrupted save game I had last night was due to FO still doing stuff for the saving when I clicked on resign and quit. I may well be worng, but if I'm not, making UI totally unresponsive while performing such actions (even when clicking on next turn) could be a good idea. Something like a pop-up "Saving, please wait" and locking off the UI until it finishes.


Off-topic (you probably know this already, so I won't open a new thread for this):
Regarding AI performance in terms of skills, it is rather good at start, at least in forcing me go combatant earlier than desired, but its struggling is hopeless once I get plasma and ship shields. By then my PP is half of them but my RP are nearly 4x theirs. Then AI becomes stupid (it sends several fleets of troop ships, spanning 5-10 turns, the first waves poorly defended, the last waves with no scort at all, that obviously crash into my unmoving fleet for 5-10 turns; keeps devoting too much industry to fleets that do nothing or die off easily; then it seems to realise that it is way behind in research since I keep finding exobot planets focused to research, but now it's too late). Now I found a Titanic Hull ship from eaxaw empire (yay), surrounded by near 100 smaller ships, at one of its main systems. Next turn I'll find out if my 20-ship chato fleet of sentient deathbringers with ultimate shielding, stealth, armour and weapons will have any loss.

Awesome game, guys. As a MoO2 and GalCiv2 player I can tell you that. Impressive work.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Problems with game speed

#11 Post by Dilvish »

Oberlus wrote:... FO 0.4.6 (freshly installed a few days ago).
...I think the corrupted save game I had last night was due to FO still doing stuff for the saving when I clicked on resign and quit. I may well be worng, but if I'm not, making UI totally unresponsive while performing such actions (even when clicking on next turn) could be a good idea. Something like a pop-up "Saving, please wait" and locking off the UI until it finishes.
That issue and many other improvements and bugfixes are already dealt with in the current builds. I would strongly suggest that you reinstall with the current weekly build; no one who makes it to these boards should still be playing the plain 0.4.6. at this point.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Vezzra
Release Manager, Design
Posts: 6095
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Problems with game speed

#12 Post by Vezzra »

Dilvish wrote:I would strongly suggest that you reinstall with the current weekly build
Or, if you can wait a few days, then 0.4.7 RC1 will most likely be available, and you can assist us in playtesting the upcoming release. :D

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: Problems with game speed

#13 Post by MatGB »

Oberlus wrote: Machine:
- Windows 10.
- i5 2500K 3.4 GHz, 16 GB RAM.
- FO 0.4.6 (freshly installed a few days ago).
That's higher spec than either my Linux desktop or my Windows laptop so you should be able to play without many issues if nothing's going wrong.
Game set up:
- Spiral 4-arm
- 200 systems.
- Starlane freq.: High
- Planet density: High
- Monsters: Low
- 7 IAs (aggressive).
No problem with any of this: I am finding the AI performs a LOT better on High planets and it will mean that there are more populated planets: each population centre generates an effects processing loop, each of these loops can take some processing time at certain points.

Having said all this, there has been a lot of refactoring on the backend over the last 6 months and we are working on the final stages of Release 0.4.7, so direct comparisons are going to be hard: I understand completely the desire to get the hang of the game with the most recent Stable version but I do strongly recommend when you start your next game installing the most recent Test version or, if it's available, RC1 for 0.4.7, because I'm fairly sure some of your problems have been solved.
Experience:
I began that game yesterday and I devoted a quite a time to reach turn 200 (still reading the different techniques/buildings/hulls/etc. and figuring out best strategies), when I noticed the relatively long times of unresponsiveness for turn changes, around 10-15 seconds (not a problem indeed). At turn 250 it was taking more than 60 seconds, plus some lag (seconds wide) began to happen in the IU (between turn-changes), at actions like clicking on a different system in the map or opening the production queue.
This ties in a bit with my experience: if lag gets bad, saving, exiting and reloading can speed things up sometimes, the difference has reduced over time (which suggests some of the root problems have been squashed) but it is still there. I rarely play through several hundred turns in one go so the build of of leakage will be less obvious. Plus, for fairly obvious reasons, replicating an issue that requires you play several hundred turns in one go is harder ;-)
Other applications (browser, libreoffice) didn't show any unresponsiveness even when alt-tabbing to them while FO was calculating next turn.
At turn 269 it was already late so I saved, waited to the save window to close and exited (resign + quit) the game.
This morning the game I saved was corrupted (it was in the folder but FO won't show it in the load or save windows). No problem since auto-save had a copy of the previous turn for me. Loaded and noticed that the time to change turns is now under 5 seconds (while yesterday it was 70-80 seconds at the time of saving). Right now I'm on turn 290 and it is still under 10 seconds.
Yeah, the savegame corruption thing was annoying, and close to exactly the solution you propose has now been implemented: it took awhile for us to work out what was happening as the time-to-save increases with size of game and when testing a feature you tend to see if it's working with a newish game, but as far as I can tell it's now resolved.
Thoughts:
It seems to me that lag increases over time way faster than the number of colonized planets, number of fleets, frequency of combats, etc.
That and the fact that exiting FO and reloading game back makes it quite smooth again makes me think that there is something odd and wrong, maybe in FO's memory management, and more specifically its data structures. I should have checked memory usage but it didn't seem to be the problem (raw RAM comsuption). I presume it has to do with temporal structures (indexes and the such) that get cluttered over time and cause very slow read operations.
This would make sense, although the backend is beyond me, there's definitely some sort of leakage and it has got less bad over the most recent cycle, it's not noticeably a memory specific issue as, like you say, RAM doesn't spike (unlike eg Firefox which still has issues on Linux after 15 years)
Also, I noticed the AI hardly groups ships into fleets. Most of the time each AI under my radars keeps the bulk of its army in a planet in a myriad of fleets composed of 1-3 ships each. If AI makes calculations for each of its fleets (instead of for each ship regardless of grouping), then I wonder if it would reduce lag to induce AI to merge fleets in same system with same aims or no aim.
It probably won't make a massive difference, for a start the AIs each run as a separate process and also you'd still want to decide if a fleet needs/wants splitting. Each ship does create an effectsloop, same as each planet, we've tried to reduce the strain this creates in this cycle by refactoring when effects are triggered but it's an ongoing process. Plus, there has been a massive AI refactoring, especially in fleet management, so some of the issue will have been reduced.
Off-topic (you probably know this already, so I won't open a new thread for this):
Regarding AI performance in terms of skills, it is rather good at start, at least in forcing me go combatant earlier than desired, but its struggling is hopeless once I get plasma and ship shields. By then my PP is half of them but my RP are nearly 4x theirs. Then AI becomes stupid (it sends several fleets of troop ships, spanning 5-10 turns,
The troop ship attack has been massively reduced this cycle, and AI stupidity also less obvious, it was incredibly annoying at times when you'd see suicidal troop fleets leading attacks, but I've not really noticed them recently.

Seriously, give the most recent Test a go, not just for the Fighters (which you can ignore if you want), but the AI improvements this cycle have been brilliant: I've lost a couple games recently, mostly my own fault but even so...
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

User avatar
Oberlus
Cosmic Dragon
Posts: 5713
Joined: Mon Apr 10, 2017 4:25 pm

Re: Problems with game speed

#14 Post by Oberlus »

Dilvish wrote:I would strongly suggest that you reinstall with the current weekly build; no one who makes it to these boards should still be playing the plain 0.4.6. at this point.
Oh, I realise that now, sorry.
Vezzra wrote:Or, if you can wait a few days, then 0.4.7 RC1 will most likely be available, and you can assist us in playtesting the upcoming release. :D
You can count on it.
MatGB wrote:for fairly obvious reasons, replicating an issue that requires you play several hundred turns in one go is harder
Indeed. If I can find time out of my behind-schedule work, I'll look in the forums for hints about compiling from source in Linux (I'm Ubuntu 12.04) and see if valgrind can figure out something here (no idea if it applies with all the python and whatnot involved).

BTW, I started my second game yesterday (Gyisache, 3 AIs maniacal, disc 100 systems, still 0.4.6) and I got pwned. There was no suicidal troop ships, combat fleets retreated when overpowered fleets were incoming... George keep its RP on par with me and his PP rocketeed. I guess I had noob luck on my first serious game, or maniacal is way better than aggressive. Whatever, AWESOME experience. I would say this AI is way better than GalCiv's and MoO2, so I feel enthusiastic about my next game (that will be for current weekly build, can't wait). Thank very much for this, guys. Kudos!
And then I read this:
MatGB wrote:the AI improvements this cycle have been brilliant: I've lost a couple games recently, mostly my own fault but even so...
Yay! can't wait till this night to play.

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: Problems with game speed

#15 Post by MatGB »

Oberlus wrote:Indeed. If I can find time out of my behind-schedule work, I'll look in the forums for hints about compiling from source in Linux (I'm Ubuntu 12.04) and see if valgrind can figure out something here (no idea if it applies with all the python and whatnot involved).
I'm on similar, one of the first things I did after I got this box was learn to do it and we updated the Wiki page based on my experiences: Compile In Linux - FreeOrionWiki
and the best forum thread to read through/post questions to if you get stuck is the one I used: FreeOrion • View topic - Compile Problem there've been changes since that thread but if you make the same mistakes I did then the solutions are there (setting up not easy, keeping going afterwards really easy, I recompile before each game and it works fine)
BTW, I started my second game yesterday (Gyisache, 3 AIs maniacal, disc 100 systems, still 0.4.6) and I got pwned. There was no suicidal troop ships, combat fleets retreated when overpowered fleets were incoming... George keep its RP on par with me and his PP rocketeed. I guess I had noob luck on my first serious game, or maniacal is way better than aggressive. Whatever, AWESOME experience. I would say this AI is way better than GalCiv's and MoO2, so I feel enthusiastic about my next game (that will be for current weekly build, can't wait). Thank very much for this, guys. Kudos!
Oh yeah, the AI is already better than in most commercially released games with similar scope that I've played, it still makes mistakes a lot but then so do I, that it's getting even better is brilliant, the game wouldn't be worth playing without the improvements made.
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Post Reply