RFC: A single source for default values (FOCS, python, stringtables, backend)

For what's not in 'Top Priority Game Design'. Post your ideas, visions, suggestions for the game, rules, modifications, etc.

Moderator: Oberlus

Message
Author
Ophiuchus
Programmer
Posts: 1821
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#46 Post by Ophiuchus »

Ophiuchus wrote: Sat Oct 17, 2020 11:59 am
Geoff the Medio wrote: Fri Oct 16, 2020 2:41 pm Try also -1.0 ?
good point
fixed in PR-3194
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12802
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#47 Post by Geoff the Medio »

I discussed the difference between primary_expr and expr in the parser here: https://github.com/freeorion/freeorion/ ... -713777942

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#48 Post by Oberlus »

Feedback on the UI, Ophiuchus (at last!).

In the Pedia pages, the named values are shown oddly:

- When not hovering, you see (e.g.) "0.5000 (per planet size)".
- When hovering: "(2.5 * 0.2000)".

That causes trouble with Pedia reading:

- Increases Target Research by 0.5000 (per population) per population.

I suggest:

- When not hovering, you see (e.g.) "0.5000".
- When hovering: "2.5 * 0.2000 (per planet size)".


The issue with the unwanted zeroes is another thing.

Ophiuchus
Programmer
Posts: 1821
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#49 Post by Ophiuchus »

Oberlus wrote: Thu Oct 22, 2020 6:39 pm Feedback on the UI, Ophiuchus (at last!).
Thanks, appreciated.

I went for the following in PR-3206

- When not hovering, you see (e.g.) "0.5000".
- When hovering: "0.5000 (per planet size: 2.5 * 0.2000)".

Also I ignore missing stringtable entries the same way as empty ones now:
- When hovering: "0.5000 (2.5 * 0.2000)".
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#50 Post by Oberlus »

A doubt:

Code: Select all

BAD_INDUSTRY_DESC
−	Bad [[metertype METER_INDUSTRY]]: 75%
Is it OK if I change that with this?

Code: Select all

BAD_INDUSTRY_DESC
−	Bad [[metertype METER_INDUSTRY]]: ×[[value BAD_INDUSTRY_TARGET_INDUSTRY_SCALING]]
It would show ×0.75 instead of 75%.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12802
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#51 Post by Geoff the Medio »

x0.75 isn't ideal, but I doubt there's an easy way to otherwise control the formatting of such a value.

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#52 Post by Oberlus »

Geoff the Medio wrote: Wed Nov 04, 2020 11:46 am x0.75 isn't ideal, but I doubt there's an easy way to otherwise control the formatting of such a value.
I already went ahead in my local and changed the fixed values in percentage for the ×[[value RATIO]].
But I could register a new named value with value 100 * 0.75, then format it as [[value PERCENTAGE]]%.
I'll do that if it's preferred.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12802
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#53 Post by Geoff the Medio »

Oberlus wrote: Fri Nov 06, 2020 9:29 amI could register a new named value with value 100 * 0.75, then format it as [[value PERCENTAGE]]%
sounds reasonable...

might have issues with how many digits to show, though?

Ophiuchus
Programmer
Posts: 1821
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#54 Post by Ophiuchus »

Geoff the Medio wrote: Fri Nov 06, 2020 9:57 am
Oberlus wrote: Fri Nov 06, 2020 9:29 amI could register a new named value with value 100 * 0.75, then format it as [[value PERCENTAGE]]%
sounds reasonable...

might have issues with how many digits to show, though?
Could go the other way. Dividing an integer by 100

Anyway in the perfect world we would have formatting flags/patterns in the stringtables
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#55 Post by Oberlus »

As commented in older posts of this thread, including macro files in stringtables does work. It just won't make calculations.
And it will break if you include macro files with "//" comments (which are currently present in some macro files).
Otherwise, it's OK to include macro files in stringtables and use those macro files as single source for default values.

And I didn't really understood it at the moment. So I was overdoing the named values thing: there are many values that do not depend on a calculation, so macro substitution is better than named values (less lookup calls).

I'm going to redo non-calculation named values as macros, I already got some working:

In stringtables/en.txt

In the CONFEDERACY policy:

Code: Select all

Increases [[metertype METER_TARGET_INFLUENCE]] on colonies disconnected from the capital by [[SUPPLY_DISCONNECTION_INFLUENCE_MALUS]].
(instead of [[value SUPPLY...]])

At the end, before the other includes:

Code: Select all

#include "../scripting/common/base_prod.macros"
In /scripting/common/base_prod.macros, remove the "//" comment at the beginning.
/Edit (missing): Add this to base_prod.macros:

Code: Select all

SUPPLY_DISCONNECTION_INFLUENCE_MALUS
'''1'''
Result:/
macros_in_stringtables.png
macros_in_stringtables.png (59.88 KiB) Viewed 77 times

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#56 Post by Oberlus »

Now, based on the above, and watching the problems that Ophiuchus is having to solve certain race conditions and whatnot in the named values manager, I think there should be a reasonably easy solution for all this that does not involve named values lookups at all, or substitutions of [[value NAMED_VALUE_NAME]] calculated values in the stringtable with tooltips and the such.

So far, all the values that we might want to include in stringtables are invariant onces a game has started, because they depend on game rules, galaxy settings or values written in macro/FOCS files. None of those change during a game, so (AFAIK) they all could be known before any FOCS effect is evaluated.

Method:

For every calculated value in a FOCS effect, such as

Code: Select all

Target.Population * 5 * [[RESEARCH_PER_POP]]
(which is hardwritten as "1 per population" in the stringtables, from 5*0.2), substitute it by

Code: Select all

Target.Population * [[THIS_TECH_NAME_RESEARCH_PER_POP]]
And include a new macro file (ignore the fancy name and pressumed folder) "calculated_values__do_not_edit.macros" which will have:

Code: Select all

THIS_TECH_NAME_RESEARCH_PER_POP
'''1'''
This "calculated_values.macros" would be recreated (by the backend) when FreeOrion client/server starts and when a game (with its own default values, game rules, galaxy settings, etc.) is set to start (pressed "Start Game"), in both cases before anything else (or particularly before any parsing), by reading and parsing a file (ignore the fancy name and pressumed folder) "/scripting/common/values_source_file__edit_this_one.macros" that would include:

Code: Select all

THIS_TECH_NAME_RESEARCH_PER_POP
'''5 * [[RESEARCH_PER_POP]]'''

#include /scripting/common/base_prod.macros
The stringtable would also include "calculated_values__do_not_edit.macros", and use "by [[THIS_TECH_NAME_RESEARCH_PER_POP]] per population" to show "by 1 per population".

So, if I wanted to increase the bonus from 1 to 2 per population, I would edit "values_source_file__edit_this_one.macros", change "5 * [[RESEARCH_PER_POP]]" by "10 * [[RESEARCH_PER_POP]]", and restart freeorion, and automatically all values would be updated in FOCS effects and stringtables, without lookups every turn.
If we had a game rule to change [[RESEARCH_PER_POP]] before starting a game (instead of being a plain macro), backend would also do the substitution when creating file calculated_values__do_not_edit.macros, so by starting a game with that value set to 0.5, so 5*0.5, the recreated file would contain

Code: Select all

THIS_TECH_NAME_RESEARCH_PER_POP
'''2.5'''
And again values in all focs files and stringtables would update.

I assume the same could be done to unify AI values sources.


Is all this possible or a good idea to replace part or all of current named values mechanics?

Ophiuchus
Programmer
Posts: 1821
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#57 Post by Ophiuchus »

Oberlus wrote: Mon Nov 16, 2020 11:18 am Is all this possible or a good idea to replace part or all of current named values mechanics?
I am not sure how this is supposed to work - possibly i did not understand what you suggest.

If i got you right you suggest using macro expansion.

You can do that today for the places where it works (so for many places where you used named values it is not strictly necessary).

It does not work for
  • static calculated values in - because macro expansion leads to "5 * 5" and not to "25", one would need to add a numeric parsing/execution step in the middle of macro expansion. the easiest would be to define a hierarchy based on file names (one file with plain values to be parsed first, one file with macros to be parsed/executed). if we do not cache macro expansion this could slow down parsing
  • calculated values based on gamerules (or AI) - would need one more full reparse on server when game is started
  • AI - because of no support for macro expansion in python
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#58 Post by Oberlus »

Ophiuchus wrote: Thu Nov 19, 2020 9:59 am I am not sure how this is supposed to work - possibly i did not understand what you suggest.
Or maybe I didn't explain it right (and certainly not briefly). Second try follows.

I'm suggesting

1. moving all FOCS expressions that involve only constants and that are used in more places than just FOCS files (i.e. stringtables or AI) to new special macro files. And substitute the expression in the original FOCS files by a new macro that will involve no calculation (will be a constant).
E.g. In ALGO_ELEGANCE.focs.txt change "0.25 * [[RESEARCH_PER_POP]]" per "[[ALGO_ELEGANCE_RESEARCH_PERPOP]]", in "single_source_values.macros" declare

Code: Select all

ALGO_ELEGANCE_RESEARCH_PERPOP
'''0.25 * [[RESEARCH_PER_POP]]'''
2. implementing a new C++ (or python?) module, executed before any parsing (so at client start and also at game start), that
- reads single_source_values.macros and the files included in it;
- perform macro substitution and calculates expression values (all operands in expressions are constants);
- write down a new file "calculated_values.macros" that has the same macros declared in "single_source_values.macros" with the expressions substituted by the final results;
- for AI, write down a "calculated_values.py" file that essentially is the python version of the macro file, for inclusion in AI python code.

3. in stringtable's description of ALGO_ELEGANCE, use "[[ALGO_ELEGANCE_RESEARCH_PERPOP]]" (instead of "0.05" or "[[value ALGO_ELEGANCE_RESEARCH_PERPOP]]") and include "calculated_values.macros" but do not include "single_source_values.macros". So macro expansion ends up being "0.05" instead of "0.25 * 0.2". Same for AI but using "calculated_values.py".


It should not slow down parsing since it is done only once before first turn of current game session.
It should involve no changes in backend parsing (the changes are in FOCS files and in the addition of that backend pre-parsing module).
Game rules in FOCS should also be read at same time than single_source_values.macros, during the pre-parsing module. But I don't think that is any kind of problem.

Ophiuchus
Programmer
Posts: 1821
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#59 Post by Ophiuchus »

There are no known performance issues with named values (else, please prove/measure). If there were perfomance issues, one could bring the runtime cost of named values down to zero by changing the implementation (i am pretty sure this is not worth the investment).

I expect for example introducing e.g. combatTarget conditions to have guesstimated two orders of magnitude higher impact on performance than named values. Nobody complained about that one.

I understand that you want this to work - and there was a sequence of issues I am solving.

I do not have the time to follow your suggestion through. Even the discussion here takes my precious time (reread my last post to see what all is involved in getting your macro approach running). I will rather finish named values, because it is going to take less time/effort and will have higher payoff.
Oberlus wrote: Thu Nov 19, 2020 11:09 am It should not slow down parsing since it is done only once before first turn of current game session.
Just as an example: This is self-contradictory. Starting a game is the time when parsing happens (for clients).
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Oberlus
Cosmic Dragon
Posts: 3186
Joined: Mon Apr 10, 2017 4:25 pm

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

#60 Post by Oberlus »

Fair enough. I know you are solving the issues. We'll see if there is any performance impact.

Ophiuchus wrote: Thu Nov 19, 2020 12:17 pm I do not have the time to follow your suggestion through. [...]
Oberlus wrote: Thu Nov 19, 2020 11:09 am It should not slow down parsing since it is done only once before first turn of current game session.
Just as an example: This is self-contradictory. Starting a game is the time when parsing happens (for clients).
Forget about slowing down parsing, I meant slowing down general gameplay performance (waiting times for players).
Lookups are done in every turn, isn't it? And the suggestion would remove the need for lookups, and the need for syncronization/mutex in the parsing (which we don't know yet if it's a significant wait or not, so it's OK to finish named values and see if it's good enough).

Post Reply