4.1 slower and slower

[hpfx]

Hello,

I tried FreeOrion v4.1 for windows (svn 5096)
It looks interresting...
But i must says that's almost impossible to play after 100 turns.
Around turn 200 it takes about 5 minutes to complete a turn !
but also User Interface is becomming so laggy (clicking on item takes 1 or 2 sec)
I'm changed graphic options to disable almost all (it's a little bit better) and I changed autosave from "every turn" to "every 10 turn"...

[Geoff the Medio]

It would probably help to play a smaller galaxy with fewer AIs. The delays and hangs are likely due to updates to the gamestate and passing of data between the server and clients which depend on the number of objects (planets, fleets, ships, buildings), so having less to process would speed things up.

[yandonman]

Is it in the plan to fix this performance problem before hitting 1.0.0, being that we are on 0.4.1 currently?

[Geoff the Medio]

Performance improvements are one of many things I'd like to get done at some point.

[yandonman]

That's a completely reasonable answer, considering the stage of development FreeOrion is at.

I would be more than happy to help with any performance testing as I have some experience in that arena, if it would help.

[Geoff the Medio]

I would be more than happy to help with any performance testing as I have some experience in that arena, if it would help.

If you can identify specific situations, or more usefully, specific techs, buildings, ship parts / hulls that lead to large performance problems, that might be helpful. Performance improvements might be possible by just modifying some content scripts.

[yandonman]

With regards to "specific situations" do you just want numbered steps?
Do you want logs?
Do you want saved game files?
Do you want ETLs?


What do you mean by large? Greater than 10 sec?

The basic advice regarding response times has been about the same for thirty years [Miller 1968; Card et al. 1991]:

• 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
• 1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.
• 10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.

[hpfx]

It would probably help to play a smaller galaxy with fewer AIs. The delays and hangs are likely due to updates to the gamestate and passing of data between the server and clients which depend on the number of objects (planets, fleets, ships, buildings), so having less to process would speed things up.

Thank you,
but you know I just played "quick start", there was not such much galaxy I think.

If you can identify specific situations, or more usefully, specific techs, buildings, ship parts / hulls that lead to large performance problems, that might be helpful. Performance improvements might be possible by just modifying some content scripts.

I think the question is not "how to optimize" performance, but more how avoiding the AI ruining the game.
In fact, when I arrived a any AI galaxy there was hundred ships waiting here.
that's maybe why game is so slow ?

regards.
note : I have samegame if needed.

[hpfx]

Is it in the plan to fix this performance problem before hitting 1.0.0, being that we are on 0.4.1 currently?

For your information, a game like ottd (train simulation) can run on 1024x1024 tiles map (even on 2048x2048) with more that 200 trains each competitors (up to 8 competitors:AI or real pleayer) with a lot of pathfinding (A*) for each vehicules without any major performance issue (and even worth it's real time)... and performance was good from firsts version already ( I remember version 0.3). then community is quite big.

you know it's very hard to keep player. I sent hours just because I wanted to complete it... one time I even felt asleep in the night.
maybe the issue does not appear when in multiplayer (without AI) then it does not affect you, but usual new comers will give up at round 100.

[yandonman]

@hpfx - I completely concur with your stance, and I feel that performance is probably one of the top 3 detractors of the game right now. The developers have also indicated that they view it as something they would like to fix. However, given where they are at in the development cycle, it's not unexpected that performance may not be addressed immediately.

I'm new to the game myself, and it's something that I'm looking to try to at least help fix in any way I can. Do you feel inclined to help as well?

[hpfx]

me myself, and it's something that I'm looking to try to at least help fix in any way I can. Do you feel inclined to help as well?

Yes, I just tried the lastest version possible I could find "svn 5285" but it's as slow as the one I tested (and I could not understand why I was not able to play with human building (except "supertester" one ?), despite a lot of research I still had "to unlock them" after 100 turn... and it's started to get slower btw)
I don't know how to help if needed.
But for me, perf issue is the major (and only major one)

note : as I can not read old savegame, I now have a game that can't build building, that's life of tester, I'll wait for next version.
note I started a new game without building supertester (that turn player type to supertester as I understood, but I wanted to stay human).
but it's another topic.
I'm ready to test, any test related to performance...

[Geoff the Medio]

With regards to "specific situations" do you just want numbered steps?
Do you want logs?
Do you want saved game files?
Do you want ETLs?

I want to know what causes performance problems, as specifically as possible. By specifically, I mean I also want to know what doesn't cause performance problems or how they could be avoided or reduced.

I don't know what "ETLs" are, but if saved games or logs or numbered steps are useful to illustrate what causes a problem, then do please include them.

What do you mean by large? Greater than 10 sec?

Whatever you consider to be problematically bad performance.

At the start of the game, things run fine (as far as I can tell), but at some point, things get very slow. What makes that happen? Does a particular tech or building being researched or produced suddenly cause performance issues? Is there some threshold of number of ships or number of owned planets that suddenly causes issues? Do certain focus settings cause problems? Is doing certain things in the UI a particular problem? Is it something the player does, or can you observe sudden drops in performance that might be traced to a change in what the AI is doing or has done? Does removing or modifying a certain bit of content from the game prevent the issues from occurring?

I could not understand why I was not able to play with human building

I'm having a hard time understanding the problem you're describing, but it sounds like you've got some issue where you can't produce any buildings. This isn't a performance issue, so I suggest making a separate Support thread about it. Include .log files from the first turn of such a game in a new post.

[yandonman]

I don't know what "ETLs" are,

xperf is the Windows performance analyzer that analyzes the performance of Windows applications. You "record" a session, in which it gathers a whole host of performance counters (CPU, disk usage, etc) on all of the stack traces of all of the calls on the system. For FreeOrion, we would filter it by the FreeOrion processes and we can see which calls are the most called or taking the most CPU. Basically, it gives developers detailed insight into what's going on in the system.

(here's MSDN's QuickStart link)

It what I used to generate these: sample 1 and sample 2.

The C++ performance developers that I know, who do this professionally for a living, swear that this tool makes performance debugging 100x easier than the "old way" of looking at logs, etc.

Whatever you consider to be problematically bad performance.

I'll note that I do ascribe to the [Miller 1968; Card et al. 1991] definition for user interactions. I'll see if I can't put them in some sort of stacked ranked order.

[hpfx]

but if saved games or logs or numbered steps are useful to illustrate what causes a problem...

a save game I have a lot.
what I can says performance decrease slowly (I mean not suddently : not after building something special)
see my save if it can help...

[Dilvish]

Those are pretty interesting profiling readouts Yandonman, thanks. In case the person wondering if the AIs were bogging down the game might still be wondering, I wanted to point out that the little 'Empires' and 'Messages' boxes show when the AI's are thinking & when they have submitted their game moves, and they are pretty darn quick.

I've also been thinking of trying to track down more bottlenecks. I'm doing my development on Linux, with KDevelop -- there are some compatible profiling tools available, I know, but does anyone have anything in particular to recommend? I'm not sure how to go about profiling a multithreaded app like this. Seems the server is probably the most important part.

I'd say we may as well put some priority to figuring this out a bit better now, or we may inadvertently put a lot of time into mechanisms that turn out to totally thrash playability.