Speeeed!

Talk about anything and everything related or unrelated to the FreeOrion project, especially Strategy Games.

Moderator: Oberlus

Message
Author
User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#16 Post by Geoff the Medio »

Nagilum wrote:I disagree, the obvious thing to do would be to remove destroyed objects.
It's not clear what you mean by this... I thought Dilvish was talking about the in-game objects list. Objects your empire knows are destroyed can be removed from this list.

Or if you mean remove them from the universe info sent to all players, that would give an empire information that it shouldn't have according to the visibility rules.
The reason I brought it up is that I'm wondering if that accumulation of ships contributes to the decrease in speed.
It likely does, but probably the number of destroyed ships is less of an issue than grown in the number of existing ships.
I still don't understand how the game can keep the CPU busy for _several seconds_ when I just select a different ship. The numbers being displayed have either been calculated during the turn or are the result of a handful of simple equations.
Here's where you're wrong or not considering everything relevant...

The slowdowns when selecting ships are generally going to be partly due to estimating populations on planets for the species with the selected ship. This is not done between turns; it's only done in the client when it needs to display that information.

There are well more than a "handful" of effects being applied to objects to determine their meter values. species.txt is almost 3000 lines long, and that's with a bunch of macros being used repeatedly to reduce the file size.

It's also a bit tricky to make sure only the necessary objects' meters get updated. It's likely re-estimating a bunch of unnecessary meters when selecting ships.

The calculations are also not necessarily simple. Much of the time is spent figuring out what objects to act on, rather than just acting on them. (This is an area where having lots of previously-visible ships in the player's view of the universe could slow things down...). The conditions that select those objects can be complicated, and if not written / scripted properly, can be quite slow to evaluate.

Another issue is just updating the UI. Apparently some systems just aren't very fast at allocating memory for a bunch of new screen widgets to store their info, so if that has to be done repeatedly to refresh the UI, it can take a long time. Until some recent changes, the biggest contributor to UI lag in many cases was allocating the memory to display the ship and fleet data in the fleets window. This has since been tweaked so the memory is only allocated when an individual panel is displayed, rather than allocating memory for all panels in the window (which could be hundreds for big fleets).

You might try closing the sidepanel before changing fleet selections, as this reduces the need for meter updating to be done to keep the UI updated.
...if I have three ships in a fleet the fleet will show "2.9" and similar for invading the game will show "7.9" when it should show "8" for example?
I never see .9s for the number of ships stat. From 1 to 10, it's always .0.
Why use floating point for integer type data in the first place?
The UI widget is built to handle floating point data, and was reused for displaying an integer value. The number of ships is (I think) calculated as an integer, but gets treated like a float when passed into the number formatting function by the widget's code.

Burkey
Space Squid
Posts: 76
Joined: Mon Dec 17, 2012 12:20 pm

Re: Speeeed!

#17 Post by Burkey »

eleazar wrote: the lag you describe is not what most people seem to experience.
I play the game (always latest build) on windows 7 64 bit, new core i3 with 4gb ram an get this exact same issue after 100-150 turns dependant on map size. The turn lag is mainly the 7/8 mb file at this stage but like this guy, just clicking between systems, fleets etc takes 5-10 seconds

User avatar
Nagilum
Release Manager, Design
Posts: 212
Joined: Thu Dec 31, 2009 3:25 pm
Location: Germany

Re: Speeeed!

#18 Post by Nagilum »

Geoff the Medio wrote:
Nagilum wrote:I disagree, the obvious thing to do would be to remove destroyed objects.
It's not clear what you mean by this... I thought Dilvish was talking about the in-game objects list. Objects your empire knows are destroyed can be removed from this list.

Or if you mean remove them from the universe info sent to all players, that would give an empire information that it shouldn't have according to the visibility rules.
I don't see the ships in main view so following that logic I should not see them in these objects in the objects Windows either.
Geoff the Medio wrote:
I still don't understand how the game can keep the CPU busy for _several seconds_ when I just select a different ship. The numbers being displayed have either been calculated during the turn or are the result of a handful of simple equations.
Here's where you're wrong or not considering everything relevant...

The slowdowns when selecting ships are generally going to be partly due to estimating populations on planets for the species with the selected ship. This is not done between turns; it's only done in the client when it needs to display that information.

There are well more than a "handful" of effects being applied to objects to determine their meter values. species.txt is almost 3000 lines long, and that's with a bunch of macros being used repeatedly to reduce the file size.
Even going through a 3k long list of modifiers and applying them will happen within a millisecond. And then the results can be cached.
Geoff the Medio wrote:It's also a bit tricky to make sure only the necessary objects' meters get updated. It's likely re-estimating a bunch of unnecessary meters when selecting ships.

The calculations are also not necessarily simple. Much of the time is spent figuring out what objects to act on, rather than just acting on them. (This is an area where having lots of previously-visible ships in the player's view of the universe could slow things down...). The conditions that select those objects can be complicated, and if not written / scripted properly, can be quite slow to evaluate.
That part I don't understand. Don't you simply get the widgetID that was selected? Or are you saying that the GUI libs need that time and FO doesn't even get the event before the GUI lib used a second or so to figure out what widget was selected?
And if that part can't be done fast enough, maybe a more static screen layout would be better?
Geoff the Medio wrote:Another issue is just updating the UI. Apparently some systems just aren't very fast at allocating memory for a bunch of new screen widgets to store their info, so if that has to be done repeatedly to refresh the UI, it can take a long time. Until some recent changes, the biggest contributor to UI lag in many cases was allocating the memory to display the ship and fleet data in the fleets window. This has since been tweaked so the memory is only allocated when an individual panel is displayed, rather than allocating memory for all panels in the window (which could be hundreds for big fleets).
Use static buffers whenever possible. Do bulk operations when adding/removing widgets?
Geoff the Medio wrote:You might try closing the sidepanel before changing fleet selections, as this reduces the need for meter updating to be done to keep the UI updated.
I'll try but when I want to invade there is no way around it.
Geoff the Medio wrote:
...if I have three ships in a fleet the fleet will show "2.9" and similar for invading the game will show "7.9" when it should show "8" for example?
I never see .9s for the number of ships stat. From 1 to 10, it's always .0.
Why use floating point for integer type data in the first place?
The UI widget is built to handle floating point data, and was reused for displaying an integer value. The number of ships is (I think) calculated as an integer, but gets treated like a float when passed into the number formatting function by the widget's code.
Then there is a bug in there :-/

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Speeeed!

#19 Post by Dilvish »

here's a thought to address the issue that stale ship objects (ones not observed recently, whose likelihood of having been destroyed unobserved increases over time) are, all agree to at least some extent, congesting the Effects calculations and various UI interactions, but without giving an empire extra, unearned, info about destroyed objects --

what if all non-static-location objects (i.e., ships with nonzero speed) that had not been observed by the respective Empire for some period of time, say 10 or 20 turns is probably as long as anyone would want stale info about a movable ship, were excluded from the normal operating Objects list/set which is the grist for many of these apparently computationally demanding operations. They could go into a 'stale objects list' that could act akin to the previous 'known-destroyed-objects' list and the corresponding actual objects could be excluded from that copy of the Universe, to streamline computations and de-clutter the UI but without improperly 'giving info' to Empires. The copy used by the Server would remain empty so as to not improperly throw off the actual Effects calculations used by the Server
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#20 Post by Geoff the Medio »

I'm reluctant to add arbitrary-seeming cutoffs like the 20 turn memory limit... It's also possible or potentially possible for just about anything to move, including systems or planets, so I'm skeptical about adding special cases for just ships/fleets to the data actually sent to clients by the server. (Client-side only display tweaks are a bit more acceptable...)

I'd also like to avoid making such changes unless it's demonstrated that it's actually necessary for significant speed improvements.

Additionally, actually removing objects from the info sent to players, even after they are destroyed, has some potential UI problems, particularly relating to sitreps form previous turns that reference those objects. Sitreps are stored with object IDs, and when a name for an object is needed in sitrep text, it looks up the object in the client-side universe, which will still contain destroyed or previously-visible objects. Without that info, sitrep errors would be shown (or hidden) instead.

Also, there already is a stale objects list (and there still are various destroyed objects lists). Stale objects are ones that an empire has previously observed and would expected to be able to see at their last known location, but which aren't currently visible there.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Speeeed!

#21 Post by Dilvish »

Geoff the Medio wrote:I'm reluctant to add arbitrary-seeming cutoffs like the 20 turn memory limit...
ok well as this is all about client side performance, it seems eminently suitable for a user setting in options, which could default to infinite, for no culling.
It's also possible or potentially possible for just about anything to move, including systems or planets, so I'm skeptical about adding special cases for just ships/fleets to the data actually sent to clients by the server. (Client-side only display tweaks are a bit more acceptable...)
So you're pointing out that a planet ghost, which I proposed to never be culled, could in fact represent a planet or system that is no longer present. That's already the case; I fail to see it as a problem at all, let alone one to do with my proposal. The choice of ship and fleet info for potential culling is not arbitrary -- they represent the vast vast majority of stale info that has become useless, unlike stale planet info.
I'd also like to avoid making such changes unless it's demonstrated that it's actually necessary for significant speed improvements.
I can understand you not wanting to commit any such changes into the main code base without actual evidence it noticeably improves things for at least some players, could be expected to marginally help or at least not hurt other players. Also, I can imagine you may be concerned that since you're the one most familiar with this aspect of the code, you might be expected to handle this -- vexing if you're far from convinced it would be a productive use of time. That would be a quite reasonable concern. To give some hard numbers to contribute to the discussion, below is a chart showing the extent of stale info in a developed game(150 stars, 12 empires, up to 200 turns), and the corresponding PP over time just for reference. Keep in mind this is a tally over all empires, so a single fleet that had at any time been observed by 5 different empires would contribute a total of 5 towards the tally. Also, this stale info did no exclude fleets on the known-destroyed-objects list; perhaps it should (I'm not clear on for what purposes that already acts as a filter against all known objects, though I think it does for at least some, like Effects) Towards the end of the run, over a third of all fleet objects were stale (having once been observed, had not again been observed for the past 20 turns). I don't really know enough about the various operations involved & their scaling complexity to know how to interpret this info well, but perhaps someone can.
Additionally, actually removing objects from the info sent to players, even after they are destroyed, has some potential UI problems, particularly relating to sitreps form previous turns that reference those objects. Sitreps are stored with object IDs, and when a name for an object is needed in sitrep text, it looks up the object in the client-side universe, which will still contain destroyed or previously-visible objects. Without that info, sitrep errors would be shown (or hidden) instead.
Well, if this were implemented according to an optional user selected cutoff then the user could be expected to know (and clearly be warned) they are making a tradeoff between performance and access to stale info. I think any such sitreps would only reference the object & its name if the object were in fact observed, right, so any problematic sitreps would inherently be rather old as well. It certainly seems like a reasonable choice to allow the user to make, and of course if we really wanted to we could plan other ways of handling that name problem.
Also, there already is a stale objects list (and there still are various destroyed objects lists). Stale objects are ones that an empire has previously observed and would expected to be able to see at their last known location, but which aren't currently visible there.
ok so this would involve some change to the stale object rules & player expectations, if the player elects to cull stale info, but I don't see it as a big change, & those rules have already been subject to plenty of change in recent history. Who really cares that much about whether an enemy ship was seen someplace 20 turns ago? That's pretty darn stale info. Suppose the ghost is 40 turns old -- keeping in mind the player's memory is unlikely to accurately log if the ghost is 10 turns old versus 40 displaying such a stale ghost seems a bit misleading, in the sense of having negative value, not in the sense of being dishonest, to me. And if you point out that the way the game dynamics go, it is rather unlikely that a 40 turn old ghost would ever still be displayed (other than large static monsters which would unaffected by the proposed change) then I'd agree with you & that just suggests to me that the info need not be retained at that point.
staleFleets.png
staleFleets.png (38.57 KiB) Viewed 14401 times
staleFleetsPP.png
staleFleetsPP.png (135.2 KiB) Viewed 14401 times
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#22 Post by Geoff the Medio »

I didn't read that whole post by Dilvish carefully, but you might want to modify the code yourself that decides what info to send to players to see if it makes much of a difference if destroyed, stale, or not-visible-for-X-turns objects are excluded. Universe::GetObjectsToSerialize is run on the server to decide what gets sent to players. Instead of just copying the empires' latest known objects map, you'd have to iterate through it and just copy the ones that have been observed in the appropriate time / aren't stale (as FO defines it) / haven't been destroyed.

User avatar
qsswin
Pupating Mass
Posts: 93
Joined: Tue Oct 18, 2011 6:48 pm
Location: UTC-5

Re: Speeeed!

#23 Post by qsswin »

Nagilum wrote:
Geoff the Medio wrote:It's also a bit tricky to make sure only the necessary objects' meters get updated. It's likely re-estimating a bunch of unnecessary meters when selecting ships.

The calculations are also not necessarily simple. Much of the time is spent figuring out what objects to act on, rather than just acting on them. (This is an area where having lots of previously-visible ships in the player's view of the universe could slow things down...). The conditions that select those objects can be complicated, and if not written / scripted properly, can be quite slow to evaluate.
That part I don't understand. Don't you simply get the widgetID that was selected? Or are you saying that the GUI libs need that time and FO doesn't even get the event before the GUI lib used a second or so to figure out what widget was selected?
And if that part can't be done fast enough, maybe a more static screen layout would be better?
I may be misunderstanding what Geoff was saying, but I think that he is referring to a problem different, and perhaps harder to solve, than the widget allocation. I believe he was referring to the fact that whenever a ship is selected, the code evaluates the max population of every planet in the system if the ship's species were to colonize it. This is so that for colony ships (and maybe outpost ships?) that number can be shown on the sidebar. That involves evaluating all the species.txt (and maybe some techs.txt) effects.

I'm slightly surprised that checking for a colonizing-type ship part on the selected ship and only evaluating if it was found wouldn't be faster, but possibly I've misunderstood and that wouldn't help.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#24 Post by Geoff the Medio »

qsswin wrote:I'm slightly surprised that checking for a colonizing-type ship part on the selected ship and only evaluating if it was found wouldn't be faster, but possibly I've misunderstood and that wouldn't help.
That's a simplification of how things work, but a similar test could probably be added to speed things up. There are many potential optimizations that haven't been implemented.

User avatar
Nagilum
Release Manager, Design
Posts: 212
Joined: Thu Dec 31, 2009 3:25 pm
Location: Germany

Re: Speeeed!

#25 Post by Nagilum »

qsswin wrote:I believe he was referring to the fact that whenever a ship is selected, the code evaluates the max population of every planet in the system if the ship's species were to colonize it. This is so that for colony ships (and maybe outpost ships?) that number can be shown on the sidebar. That involves evaluating all the species.txt (and maybe some techs.txt) effects.

I'm slightly surprised that checking for a colonizing-type ship part on the selected ship and only evaluating if it was found wouldn't be faster, but possibly I've misunderstood and that wouldn't help.
Yeah, that would be the first step. The second would be to cache the results.
I'm still a bit sceptic though that this evaluation would take so much time. If you have to do 50 instructions per lookup you end up with 150k instructions. If the average instruction takes 2 cycles that would be 300k cycles with 2GHz CPU that would be .00015 seconds.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#26 Post by Geoff the Medio »

Nagilum wrote:If you have to do 50 instructions per lookup...
There are almost definitely a lot more than 50 instructions per lookup, whatever you mean by "lookup", and you seem to not be considering all of the time spent getting data to process, both in terms of searching for the data in containers, and because the data's not all going to be in CPU registers (or even CPU cache) and ready for CPU instructions to manipulate.
...cache the results.
Planet target population estimate caching is one of many possible optimizations that aren't implemented yet. Actually doing it would be a bit tricky though; even if a bunch of assumptions are made, there's no easy place to put the code to store and check for and restore that info or generate it when not already available. Maybe in SelectedShipsChanged at the end instead of calling UpdateMeterEstimates it could fit. The necessary assumptions that nothing else can change the relevant meters might come back to cause problems, though, and there'd still be some pauses the first time things get updated before results can be cached.

Feel free to suggest ideas, but lack of them is not the limiting factor on getting optimizations implemented.
Geoff the Medio wrote:
qsswin wrote:...checking for a colonizing-type ship part on the selected ship and only evaluating if it was found...
That's a simplification of how things work, but a similar test could probably be added to speed things up.
To elaborate, the way things work now is that whenever the selected ships (may be more than one) change, meter estimates are redone. To limit the number of reestimates, the code would need to do a more complicated checking to make sure that the change in selected ships didn't change the species that would be use for colonizing, and would then need to propagate a different signal so that the called code would know about the ship selection change but that it also did so in a way that doesn't require a population reestimate. Or the called code could make that decision, and the single signal about selected ships changing could remain. How exactly to implement that is a bit tricky, as the code that decides what species to use for colonizing is probably not designed to tell other code what species would be selected in a helpful way for doing this, and actually figuring out what species are selected itself may be a complicated process...

User avatar
Foocaux
Space Squid
Posts: 78
Joined: Sat Jul 26, 2014 7:14 am

Re: Speeeed!

#27 Post by Foocaux »

I'm afraid the situation in .43+ (SVN 7278), though better than in .43 itself, is still somewhat similar to what Nagilum mentioned already.

I've got an older laptop (celeron, 3GB of ram, windows 7) and I started a standard game (150 star galaxy, 8 AIs) the game is now at turn 300 with 550 total ships and 2 surviving AIs out of the initial 8.

I'm now running FO .43+ to test the latest bugfixes, using a .43 save game, which seems to load fine.

The results? The UI is still lagging, unfortunately.
In .43 I was trying to create a new ship design but renaming a ship design was basically impossible, tapping any key for the briefest instant would always produce at least 2keystrokes on screen. This also applied to the backspace & delete keys.
In .43+ things are better already, and I can actually type normally. However, the cursor lags 1 to 2 seconds behind the actual cursor, and the same applies with all production & research screens - right-clicking a production item to bring up its menu (top of queue, bottom of queue, remove) does take about a second, while on a freshly started game (and about 50 ships added separately to the production queue itself) there is no perceptible delay.

On a faster machine these problems wouldn't be that noticeable, of course. While I am going to get a faster laptop at some point in the future, I'm also hoping to try some 'slightly' bigger games ( 1000 stars, 24 AIs) at some point, which might well bring FO's UI to its metaphorical knees.

Has any of the devs had a look at profiling the code when FO is running a turn 300 game? This would definitely highlight the major bottlenecks in the code: almost by definition, these bottlenecks are found in totally unexpected places, and even a tiny efficiency increase in very often used calls does wonders for older machines, like mine! :)

Cheers,

Mike.
The small print: Any code contribution I make to FreeOrion is made under GPL v2.0, any graphic contribution is under CC-by-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Speeeed!

#28 Post by Geoff the Medio »

Foocaux wrote:Has any of the devs had a look at profiling the code when FO is running a turn 300 game?
Probably not exactly 300 turns, but various threads have discussed profiling results.

viewtopic.php?f=9&t=8657&p=68121&hilit=profiling#p68121
viewtopic.php?f=9&t=8546&p=66273&hilit=profiling#p66273
viewtopic.php?f=9&t=8545&p=66253&hilit=profiling#p66253
viewtopic.php?f=28&t=7783&p=63798&hilit ... ing#p63798

Given your description, it sounds partly like you have rendering / FPS issues. Make sure to turn off the galaxy gas and background starfields in the options. Then try setting a low FPS limit, and try enabling / disabling "optimized" rendering.

User avatar
Foocaux
Space Squid
Posts: 78
Joined: Sat Jul 26, 2014 7:14 am

Re: Speeeed!

#29 Post by Foocaux »

Thanks for the suggestions, quite a few things I didn't think of trying there.

I just turned gas & background starfields off, set a 20 FPS limit, though I couldn't find where the optimized rendering toggle was. In 0.4.3+ the results are just about the same as before:

No real lagging at turn 1, even when moving 30+ different ships up and down in the production queue / ditto for 20+ research projects in the research screen.

However, at turn 300 lagging is evident everywhere, including the ship design screen: I just drag & dropped a mass driver 1 inside a basic small hull at turn 300 and it took about 2 seconds for FO to notice. At turn 1 (I did menu > resign > quick start) the same operation takes an instant (less than 200ms I'd guess)

However, all is not well with my 0.4.3 to 0.4.3+ savegame transplant: I failed to notice that at first, but at the beginning of each turn I do get the following message
AI 4: AI Error: AI script error in "generateProductionOrders": "list index out of range"
it might affect the UI, somehow.


I'll try and read more about cami's investigations asap, however, it does look like he just profiled the sever Daemon rather than the client/UI program, which might well be where this one thing comes from.

My gut feeling (almost always wrong, but hey, might as well... :) ) is that the UI might be calling the Daemon way too often, unnecessarily so in the case of the ship design screen.

Thanks anyway for such a great program. This problem notwithstanding, it's already much better than a lot of for-profit 4x games out there. Time permitting, I'll try & help if I can, though it'll have to wait till winter, the weather it simply too good now to be stuck in front of a computer the rare free time I get! :)
The small print: Any code contribution I make to FreeOrion is made under GPL v2.0, any graphic contribution is under CC-by-SA 3.0

Post Reply