FreeOrion

Forums for the FreeOrion project
It is currently Sun Dec 17, 2017 11:07 pm

All times are UTC


Forum rules


Always mention the exact version of FreeOrion you are testing.

When reporting an issue regarding the AI, if possible provide the relevant AI log file and a save game file that demonstrates the issue.



Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2
Author Message
PostPosted: Sun Apr 10, 2016 8:29 am 
Offline
Space Krill
User avatar

Joined: Thu Apr 07, 2016 9:26 pm
Posts: 8
Made an interesting test:
In the above mentioned savegame, I scrapped all my 1.72k ships. The time for a single focus change went down from 35s to ~6s.
Still a little long imho. And it's pretty hard to play w/o ships ;)


Top
 Profile  
 
PostPosted: Sun Oct 02, 2016 5:12 pm 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4309
Location: Sol III
Sorry for getting back to this after half a year, but RL got in the way, after that the release, and only now I got a few moments to dig through old, unfinished discussions.

Don't know if you're still here and listening, but in case you are, here a very delayed reponse...
Duke wrote:
Vezzra wrote:
All those effects are scripted in the content scripts

I hope those scripts are somehow precompiled at startup/load time ?
Yes.
Quote:
Unfortunately I don't have the time atm to set up a full IDE for FO and get familiar enough with the code to make qualified suggestions. But if you can give me handful of important filenames (where the n:n handling happens), I will take a glance or two at github.
Huh, unfortunately I'm not familiar with that part of the codebase, one of the other (more knowledgeable) devs need to chime in here. AFAIK the Condition.h/cpp and Effect.h/cpp files in the universe folder should contain the code sections you're looking for, but I likely am missing something...
Quote:
btw if we are talking about design and code, are we still in the right forum ?
Once we go into more detail, we should probably open a thread in the programming subforum.


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 3:17 am 
Offline
Space Kraken

Joined: Mon Nov 16, 2015 5:57 am
Posts: 105
Vezzra wrote:
Unfortunately, it's also extremely difficult to make improvements to this issue. There have been several attempts at such improvements...

The fundamental problem, as far as I understand, is on the design level of the effects system.
Each object in the game can have effects attached to it, and those effects can potentially affect each object.
Techs also have effects attached which also can affect each game object.
Game objects are ships, fleets, buildings, planets, systems. All those effects are scripted in the content scripts...


How is the problem of circular logic avoided then ? A effects B and if B is effected it increases its effect on A which then increases A's effect on B and so on. Priority ?

Quote:
...However, each "effectsgroup" needs to evaluate its "scope" (that's the set of object it will affect) each time it's executed, and this scope always starts with the entire set of all objects.

Does this mean if we have less effect groups overall it is faster ?
Does this also mean that refactoring pre-existing scripts to be more efficient is basically equal to a speed gain.

Quote:
Which is why the order of conditions for scopes in the content scripts matters a lot, because some conditions evaluate a lot faster than others, and you'd want to reduce the "result set" of a scope condition with fast conditions before the more expensive ones get applied.

So this is the idea of Implicitly trimming down the affected scope of a large set of objects.
Instead of Explicitly adding what is to be affected by adding it to a empty scope ?

Quote:
Each time you e.g. change the focus of a colony, all the effects of all objects need to be re-evaluated, because there is no way to safely exclude anything for which you can safely assume that it won't be affected (directly or indirectly) by a focus change.

This will probably also tell you how difficult it is to make the effects execution implementation multi-threaded (basically, you'll have to set up an execution graph first, with all the interdependencies resolved).


Quote:
I've often wondered if there is a way to redesign the fundamental effects system in a way that would allow for more efficient optimization, but so far I've been coming up blank. I'm not the most proficient coder around here, though.


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 3:51 am 
Offline
Creative Contributor
User avatar

Joined: Fri Jun 28, 2013 11:45 pm
Posts: 3295
Note: my answers are my current understanding, I'm still not great at this and thus I may be wrong, please correct me if you know better, but answering will help me understand it better.
xlightwavex wrote:
How is the problem of circular logic avoided then ? A effects B and if B is effected it increases its effect on A which then increases A's effect on B and so on. Priority ?

Multiple passes working out what affects what: you can see this with Sitreps, which sitreps will trigger is worked out on an early pass whereas the actual effects later on, so sometimes a sitrep tells you something slightly off (eg: a different ship lost its fuel from a dampening cloud, a drydock gave a partial repair but the damage control techs had already fixed it fully, a drydock repaired ship X but the stargate had moved it to location Y where there was no drydock).

Priority is a recent introduction that we still haven't applied to everything but it makes it a lot easier to control what's actually going to happen.
Quote:
Does this mean if we have less effect groups overall it is faster ?

Not by a huge amount per group, but yes, if you create your own default folder with almost nothing in it the game will play faster.
Quote:
Does this also mean that refactoring pre-existing scripts to be more efficient is basically equal to a speed gain.

Yes, this is an ongoing project (see priority being new, above), and it sometimes takes someone to notice a script being inefficient and tweaking it. Sometimes the actual change was for a completely different reason but ended up massively improving efficiency, eg: we recently reworked and relocated the population bonus scripts using the priority system, the objective was that the icons would show properly in the tech window. While doing it, it was noticeable that the game sped up substantially and it was due to the effects not being in the species folder (which is used a lot for virtually every effect) but now into their source location which isn't processed anywhere near as much. After working this out we moved as many other effects as possible out and late/large game lag has been massively reduced.
Quote:
So this is the idea of Implicitly trimming down the affected scope of a large set of objects.
Instead of Explicitly adding what is to be affected by adding it to a empty scope ?
Beyond me, someone?

_________________
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 8:25 am 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12045
Location: Munich
Evaluating the scope of an effectsgroup means finding which objects in the universe match the scope condition. In most cases, this involves applying a series of conditions to narrow down an initial list of candidate objects, within an And condition at the top level, with several other conditions contained in it.

In all cases, when evaluating a condition, some initial set of objects to consider is required. By default, or if nothing more specific has been implemented, this can be the whole universe. But when possible, the initial set is more specific: If a condition is just "Ship" then, it will start with just ship objects. If it's "InSystem id = Source.SystemID", it will look up the system object with the specified ID, then add all the objects contained by that system to the initial set. If it's "Source", it just adds the source object.

After getting the initial candidates set, all the conditions within an And condition are applied in order to those candidates, to narrow down the matches. Depending on the order they are scripted, this can be faster or slower. If you had an effect in a species script that should only work on the source ship, you could write "scope = And [ Ship Source ]" or "scope = And [ Source Ship ]". The latter should be a lot faster, because it just checks one object - the source - to see if it's a ship, rather than checking all ships to see if they are the source.


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 1:47 pm 
Offline
Vacuum Dragon
User avatar

Joined: Sun Sep 25, 2011 2:51 pm
Posts: 500
This begs the question, shouldn't the scope be (stable) sorted after parsing to improve efficiency? At least the "set Source as the first condition in an And" optimization.

_________________
[...] for Man has earned his right to hold this planet against all comers, by virtue of occasionally producing someone totally batshit insane. - Randall Munroe, title text to xkcd #556


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 1:49 pm 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4309
Location: Sol III
xlightwavex wrote:
How is the problem of circular logic avoided then ? A effects B and if B is effected it increases its effect on A which then increases A's effect on B and so on.
AFAIK all the scope conditions of all effectsgroups (which determine the set of universe objects that will be affected by a effectsgroup) are evaluated before any of the effects of all the effectsgroups are executed. Which means that if the effects of effectgroup A change which objects will match the scope condition of effectsgroup B this doesn't matter, as by the time the effects of effectsgroup A are executed/applied, the scope condition of effectsgroup B has already been evaluated and the set of object effectsgroup B will applied to determined.

However, that only applies to the effects execution of a single turn. When considering multiple turns, such circular effects are not avoided. So you can implement an effectsgroup that applies changes to an universe object A, which in turn will "increase" (in whatever form) the effects of another effectsgroup that affects an universe object B. And have the changes to B "increase" the effects of our first effectsgroup.
Quote:
Does this mean if we have less effect groups overall it is faster ?
Potentially. Depends on what you mean by having less effectsgroups. The number of effectsgroups in the content script isn't the factor here, but the number of effectsgroups that will be executed. One effectsgroup can be attached to a lot of objects, so that means if you have coded p ten effectsgroups in your content scripts, but each of them is only attached to one universe object, or one tech, and each one of them will therefore only be executed once each turn, you'll need less effects processing than if you only have one effectsgroup, but this one attached to, lets say 100 universe objects (because this same effectsgroup will have to be executed 100 times).

Of course this simple example assumes that all these effectsgroup need roughly the same time to be processed, but I think you get the picture.
Quote:
Instead of Explicitly adding what is to be affected by adding it to a empty scope ?
As Geoff already pointed out, you need to determine a initial set to apply "and" conditions, which is what you usually have: you want to apply some effects to e.g. all objects that are ships AND in a certain system AND belong to a certain empire.


Top
 Profile  
 
PostPosted: Tue Oct 04, 2016 2:06 pm 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12045
Location: Munich
em3 wrote:
...shouldn't the scope be (stable) sorted after parsing to improve efficiency? At least the "set Source as the first condition in an And" optimization.
There are few cases where this is actually done / needed. It's more of a useful example than a practical problem.


Top
 Profile  
 
PostPosted: Wed Oct 05, 2016 12:45 pm 
Offline
Space Kraken

Joined: Mon Nov 16, 2015 5:57 am
Posts: 105
Quote:
as by the time the effects of effectsgroup A are executed/applied, the scope condition of effectsgroup B has already been evaluated and the set of object effectsgroup B will applied to determined.

I was more so asking about the order of effects execution when speaking of non commutative effects.

e.g.
you have 1 planet with 5 industry total for your empire.

Effect A adds +5 industry to all planets then 5+5 = 10
Effect B multiplies industry by 2

5+5 * 2 = 20
but in opposite order 5 * 2 + 5 = 15.

Priority then controls when a effect is executed im guessing ?

Quote:
you could write "scope = And [ Ship Source ]" or "scope = And [ Source Ship ]". The latter should be a lot faster, because it just checks one object - the source - to see if it's a ship, rather than checking all ships to see if they are the source.


so

InSystem id = Source.SystemID
ship

Is better late game when there is a lot of ships then.

ship
InSystem id = Source.SystemID

Still a little confused though on how things are processed internally like, if the ships check for effects or if the effects check the ships to say what is looped thru as a executing list each turn. Objects or Effects or both.


Top
 Profile  
 
PostPosted: Wed Oct 05, 2016 1:14 pm 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4309
Location: Sol III
xlightwavex wrote:
Priority then controls when a effect is executed im guessing ?
Exactly.
Quote:
Still a little confused though on how things are processed internally like, if the ships check for effects or if the effects check the ships to say what is looped thru as a executing list each turn. Objects or Effects or both.
AFAIK effects check the ships, so to speak. If I understand correctly, internally the condition "Ship" will initialize your result set with all the ships in the universe. As such a list is maintained internally, that's a very quick operation. Same applies to all universe object "types" (planets, buildings, etc.).


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2

All times are UTC


Who is online

Users browsing this forum: Bing [Bot] and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group