Page 3 of 6

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sat Sep 05, 2020 10:39 am
by Ophiuchus
From the PR discussion, some questions about scope kreep up
Geoff the Medio wrote: Sat Nov 16, 2019 2:45 pm 1) A way to look up a scripted value in another script, in a stringtable entry, or in the AI code
For those not following the PR, the backend has three separated registries for named ValueRef<Integer>, ValueRef<Double>, and ValueRef<Whatever>. And it seems more and more that only the first two are really necessary.

The scope is currently value refs which do not depend on the context (i.e. you do not need an object/a set of objects to evaluate it). So it is basically primitive values, game rule values and arithmetic calculations based on those.
Universe-variant or object-variant value refs could also be supported in principle without too much hassle if necessary (we would mainly need to provide default values or contexts to those). What is not possible without much effort are value refs which depend on top level content (i think that are those where you use RootCandidate) because valuerefs are non-copyable and you can set the top content only once and named value refs are allowed to be used in multiple contexts (having different top level content).

So, scripters, and AI developers, speak up! Do you have use cases to register and lookup non-numerical value refs?
Geoff the Medio wrote: > I think presently there won't be a `std::string` version available, since it's not explicitly instantiated...?
A string version is currently also not supported in the parsers at all. And we have no string calculations besides macro composition. And the results of macro composition are also available in stringtables already. Do you have a use case for a string version?

I added PlanetEnvironment and PlanetType versions to the NamedValueRefParser for testing the generic registry and found it is really hard to come up for a use case of those. Such a valueref returns an enum, but what inputs could lead to calculation of those (LocalCandidate.NextBetterPlanetType? but that is not context-invariant).

Currently the main case for named value refs are calculations based on game rules because these are not known at scripting time and have a sensible default value to use if the game is not started.

I also had a look at AIDependencies and unsurprisingly most of it in the end define numerical values which are used in predicting effects of buildings/techs/species/specials/hulls/parts. For the more complex parts one would need to analyse (or have well-known/named) effects after scripting - more support for value refs does not really help.

I think in the end only the numerical versions are really necessary because only those have meaningful (and invariant) calculations and results. Or we extend in future the class of use cases (e.g. supporting object-variant value refs and supplying a default object) - but i do not see the necessity for that yet.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sat Sep 05, 2020 11:21 am
by Geoff the Medio
Ophiuchus wrote: Sat Sep 05, 2020 10:39 am...value refs which depend on top level content (i think that are those where you use RootCandidate) because valuerefs are non-copyable and you can set the top content only once and named value refs are allowed to be used in multiple contexts (having different top level content).
RootCandidate isn't related to SetTopLevelContent.

RootCandidate is just the object potentially being matched by a whole Condition tree. So if you have a condition that matches systems that contain (Ships with no owner), the RootCandidate would always be one of those systems, but within subconditions, it might be testing ships or other objects that aren't (or are... it doesn't matter) eventually going to be matched by the top level condition, but which need to be identified for the top level condition to determine its result.

SetTopLevelContent is used to set stored data in a ValueRef (which might be inside a condition or effect or another ValueRef) to track what bit of content (not UniverseObject) the ValueRef is used within. If you have a ship part that has an effect that acts differently depending how many copies of the part a ship design has, you might have a condition in a statistic valueref that calculates how many of that part are present in the design. To do that, you need the name of the part to test for, which would be the "top level content" that is set and remembered.
Ophiuchus wrote: Sat Sep 05, 2020 10:39 amI think in the end only the numerical versions are really necessary because only those have meaningful (and invariant) calculations and results. Or we extend in future the class of use cases (e.g. supporting object-variant value refs and supplying a default object) - but i do not see the necessity for that yet.
Seems reasonable.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sun Sep 06, 2020 10:10 am
by Ophiuchus
Geoff the Medio wrote: Sat Sep 05, 2020 11:21 am SetTopLevelContent is used to set stored data in a ValueRef (which might be inside a condition or effect or another ValueRef) to track what bit of content (not UniverseObject) the ValueRef is used within.
Thanks for the clarification. As RootCandidate checks for the top level objects it would be okay to reuse named value refs there and only the TopLevelContent-variant value refs are the complicated case.

A side question: Would it be possible to move the TopLevelContent into the scripting context (or similar) instead of putting it in every condition/value ref? It seems this always gets pushed down to the contained conditions/value refs anyway?

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Wed Sep 09, 2020 6:31 pm
by Geoff the Medio
Ophiuchus wrote: Sun Sep 06, 2020 10:10 amA side question: Would it be possible to move the TopLevelContent into the scripting context (or similar) instead of putting it in every condition/value ref? It seems this always gets pushed down to the contained conditions/value refs anyway?
I think the calls to TopLevelContent all just call their contained condition/ref's own TopLevelContent function, until it reaches a Constant<std::string> which stores it, and returns it if the string stored is "CurrentContent".

It doesn't really make sense to pass it with the ScriptingContext though, as it's not really part of the context... And if it was somehow used as something that varies with context, it wouldn't be constant anymore, so constant expression trees wouldn't be able to cache their "constant" value.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Thu Sep 10, 2020 10:02 am
by Ophiuchus
Geoff the Medio wrote: Wed Sep 09, 2020 6:31 pm
Ophiuchus wrote: Sun Sep 06, 2020 10:10 amA side question: Would it be possible to move the TopLevelContent into the scripting context (or similar) instead of putting it in every condition/value ref? It seems this always gets pushed down to the contained conditions/value refs anyway?
I think the calls to TopLevelContent all just call their contained condition/ref's own TopLevelContent function, until it reaches a Constant<std::string> which stores it, and returns it if the string stored is "CurrentContent".

It doesn't really make sense to pass it with the ScriptingContext though, as it's not really part of the context... And if it was somehow used as something that varies with context, it wouldn't be constant anymore, so constant expression trees wouldn't be able to cache their "constant" value.
The use case for CurrentContent is inside of macros, where for each use site a different top level content exists. For each top level content (e.g. "SP_HUMANS", "SP_TRITH"..) a whole effect/condition/valueref tree gets created.

For named-in-middle value refs i have a kind-of-sensible definition (which might bite some scripters sometimes though): the top level content name of the surrounding code (e.g. "SP_HUMANS", "SP_TRITH"..) is also applied and stays fixed for the named value ref, even if that is used in code with a different top level content (e.g. if you reference it in a ship hull, it still references "SP_HUMANS"). This also makes sense for most cases as a valueref working for a species not necessarily works for a ship hull. If you need different instantiations you could add the CurrentContent to the name definition (similar to how i create different named value refs for the different fuel tech kinds SHP_ANTIMATTER_TANK_EFFECT_MULT for SHP_ANTIMATTER_TANK tech and so on).

For the top level defined named value refs i could throw an exception if someone uses CurrentContent. After parsing I would SetTopLevelContent("THERE_IS_NO_TOP_LEVEL_CONTENT") and let ValueRef::Constant throw an exception if it is set to "CurrentContent".

If I think about it, CurrentContent should may be rather be part of macro resolution and not part of conditions/effects/valuerefs at all. We could remove that strange cascade of SetTopLevelContent and the ValueRef::Constant handling. It would increase the places where you can use CurrentContent which might be useful. Is the top level content known when macro expansion happens so it would be possible to add something like [[[TOPLEVEL_CONTENT]]] which gets expanded to the top level content name?

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Fri Sep 11, 2020 9:47 am
by Geoff the Medio
Ophiuchus wrote: Thu Sep 10, 2020 10:02 amFor named-in-middle value refs i have a kind-of-sensible definition (which might bite some scripters sometimes though): the top level content name of the surrounding code (e.g. "SP_HUMANS", "SP_TRITH"..) is also applied and stays fixed for the named value ref, even if that is used in code with a different top level content
Sounds reasonable that, if a ValueRef is defined within some content (Species, Tech, BuildingType), it will have its top-level-content set to the content it is in, even if it is looked up from elsewhere.
For the top level defined named value refs i could throw an exception if someone uses CurrentContent. After parsing I would SetTopLevelContent("THERE_IS_NO_TOP_LEVEL_CONTENT") and let ValueRef::Constant throw an exception if it is set to "CurrentContent".
I'd not throw an exception at evaluation time. If at all, then at parse time, or whenever SetTopLevelContent is called. But more likely I'd just log an error then.
CurrentContent should may be rather be part of macro resolution and not part of conditions/effects/valuerefs at all.
Macros don't know anything about conditions, effects, or valuerefs. Macros replace text... that's it. They can be used anywhere in the FOCS text, including creating the text that sets up the syntax that will later be parsed into various kinds of content. But it's not known what is being parsed until after the macro expansions are all done and the parser is run on the result.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Wed Sep 30, 2020 12:48 pm
by Ophiuchus
PR got merged

This adds
  • a configuration file `default/scripting/common/named_values.focs.txt` for defining named value refs
  • a `NamedInteger` definition in that file that takes a `string` name and a `ValueRef<int>` value as parameter.
  • a `NamedReal` definition in that file that takes a `string` name and a `ValueRef<double>` value as parameter.
  • a `NamedEnvironment` definition in that file that takes a `string` name and a `ValueRef<PlanetEnvironment>` value as parameter. (this and the following NamedPlanetType are mostly proof-of-concept how this works with enums).
  • a `NamedPlanetType` definition in that file that takes a `string` name and a `ValueRef<PlanetType>` value as parameter.
  • a encyclopedia `[Test - Named Value Refs]` entry which lists all the named value refs in the game (name/ID, registration type, stringtable entry, calculation, calculated value, and pattern of invariances)
Also in normal focs and macro files one can use:
  • `NamedRef` ValueRefs called `NamedRealLookup`, `NamedIntegerLookup` in FOCS which takes a string name as parameter. This can be used to look up and eval a registered value ref.
  • a name-in-the-middle-ValueRef definition named `NamedReal` which takes a `string` name and a `ValueRef<double>` value as parameters. This registers the given value ref and inserts a `NamedRef`instead.
  • a name-in-the-middle-ValueRef definition named `NamedInteger` which takes a `string` name and a `ValueRef<integer>` value as parameters. This registers the given value ref and inserts a `NamedRef`instead.
Currently use is showcased for fuel and reinforced hull techs. The named_values.focs.txt is used for the reinforced hull bonus (although that tech should use a name-in-the-middle valuref as that tech is the single place where the value is used). The name-in-the-middle functionality is shown for fuel tech in `fuel.macros`-using named_values one would have to make an explicit entry for each of the techs.

Code: Select all

default/scripting/techs/ship_parts/damage_control/SHP_REINFORCED_HULL.focs.txt:
  effects = SetMaxStructure value = Value + NamedRealLookup name = "SHP_REINFORCED_HULL_BONUS"

default/scripting/common/named_values.focs.txt:
  NamedReal name = "SHP_REINFORCED_HULL_BONUS" value = (5 * [[SHIP_STRUCTURE_FACTOR]])

default/scripting/techs/ship_parts/fuel/fuel.macros:
  SetMaxFuel value = Value + ((NamedReal name = "@[email protected]_MULT" value = @[email protected]) * (PartsInShipDesign Name = "FU_BASIC_TANK" design = Target.DesignID))
Registered value refs can be shown via

Code: Select all

[[value REGISTERED_VALUEREF_NAME]]
in encyclopedia/stringtables (e.g.

Code: Select all

[[value SHP_REINFORCED_HULL_BONUS]]
).
Also one can show these in sitrep messages (e.g. `[[value SHP_REINFORCED_HULL_BONUS]]` or sitrep tags in stringtables: `%value%`, `%value:tagnamedifferentfromvalue%`, ) with a mouseover effect. The mouseover shows the `ValueRef->Description`.

Note: Currently one also needs to define the relevant sitrep entry (e.g. `SHP_REINFORCED_HULL_BONUS`) - else you get errors. This might be the right thing. I use it to give some textual extra info which might be the wrong thing.

OPTIONS_DEFAULT_TOOLTIP_COLOR and OPTIONS_ROLLOVER_TOOLTIP_COLOR define the colors used for showing the tags.

AI support is not implemented yet.

TL;DR
we can now use calculated values in pedia entries and sitrep messages

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Wed Sep 30, 2020 3:06 pm
by Oberlus
Marvellous and super useful. Thank you very much. Still few time to test it with release candidate in testing, but it's next on the list.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Fri Oct 09, 2020 11:09 am
by Oberlus
This doesn't work as expected:

Code: Select all

(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = 2 * [[TROOPS_PER_POP]])
The value gets calculated as 2 (shown as "2.000000 (2.00)" in the pedia page).

You need to enclose the expression for the value in parenthesis:

Code: Select all

(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = (2 * [[TROOPS_PER_POP]]))
This gives us "0.400000 (2.00 * 0.20)".

Also, are we forced to use NamedReal in any Set<meter> effect?

Code: Select all

default/scripting/techs/defense/Garrison.focs.txt:18:52: Parse error.  Expected real number expression here:
                Planet
                OwnedBy empire = Source.Owner
            ]
            stackinggroup = "GARRISON_1_TROOPS_STACK"
            priority = 110
            effects = SetMaxTroops value = Value + (NamedInteger name = "GARRISON_1_MAXTROOPS_EFFECT" value = 6)
                                                    ^
                accountinglabel = "DEF_GARRISON_1"
    ]
    graphic = "icons/tech/troops.png"
It works when using NamedReal (or NamedRealLookup if I define it in named_values.focs.txt), but not with NamedInteger (or NamedIntegerLookup).

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sat Oct 10, 2020 8:09 pm
by Ophiuchus
Oberlus wrote: Fri Oct 09, 2020 11:09 am This doesn't work as expected:

Code: Select all

(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = 2 * [[TROOPS_PER_POP]])
The value gets calculated as 2 (shown as "2.000000 (2.00)" in the pedia page).

You need to enclose the expression for the value in parenthesis:

Code: Select all

(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = (2 * [[TROOPS_PER_POP]]))
This gives us "0.400000 (2.00 * 0.20)".
Sigh. I thought that the NamedReal would bind greedily, will have a look if i can enforce an order there, can you please open an issue?

Code: Select all

# when writing this:
(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = 2 * [[TROOPS_PER_POP]]) 
# this is intended:
(NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = (2 * [[TROOPS_PER_POP]])) 
# and not this:
((NamedReal name = "GARRISON_2_MAXTROOPS_EFFECT" value = 2) * [[TROOPS_PER_POP]]) 
Oberlus wrote: Fri Oct 09, 2020 11:09 am Also, are we forced to use NamedReal in any Set<meter> effect?
...It works when using NamedReal (or NamedRealLookup if I define it in named_values.focs.txt), but not with NamedInteger (or NamedIntegerLookup).
Yes, meters are double value refs, so one has to use NamedReal.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sat Oct 10, 2020 10:34 pm
by Oberlus
Issue #3188 open.

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Fri Oct 16, 2020 8:43 am
by Oberlus
More on this, maybe related:

Code: Select all

default/scripting/policies/CONFEDERATION.focs.txt:56:117: Parse error.  Expected real number expression here:
                OwnedBy empire = Source.Owner
                Population low = 0.001
            ]
            priority = 80
            effects = [
                SetTargetHappiness value = Value + (NamedReal name = "PLC_CONFEDERATION_MAXHAPPINESS_EFFECT" value = -1)
                                                                                                                     ^
Enclosing the value in parenthesis solves it:

Code: Select all

(NamedReal name = "PLC_CONFEDERATION_MAXHAPPINESS_EFFECT" value = (-1))

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Fri Oct 16, 2020 2:41 pm
by Geoff the Medio
Try also -1.0 ?

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Sat Oct 17, 2020 11:59 am
by Ophiuchus
Geoff the Medio wrote: Fri Oct 16, 2020 2:41 pm Try also -1.0 ?
good point

Re: RFC: A single source for default values (FOCS, python, stringtables, backend)

Posted: Mon Oct 19, 2020 10:00 am
by Oberlus
Ophiuchus wrote: Sat Oct 17, 2020 11:59 am
Geoff the Medio wrote: Fri Oct 16, 2020 2:41 pm Try also -1.0 ?
good point
It didn't work:

Code: Select all

default/scripting/policies/CONFEDERATION.focs.txt:56:115: Parse error.  Expected real number expression here:
                OwnedBy empire = Source.Owner
                Population low = 0.001
            ]
            priority = 80
            effects = [
                SetTargetHappiness value = Value + (NamedReal name = "PLC_CONFEDERATION_MAXHAPPINESS_FLAT" value = -1.0)
                                                                                                                   ^