TurnTechResearched not working the first turn

Programmers discuss here anything related to FreeOrion programming. Primarily for the developers to discuss.

Moderator: Committer

Post Reply
Message
Author
Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

TurnTechResearched not working the first turn

#1 Post by Ophiuchus »

As far as i can see FOCS TurnTechResearched condition does not work for the turn in which the tech gets researched (it starts working one turn afterward).

I guess is that on a turn when a tech gets researched not all of its effects are applied (?).
I think that normal effects happen and afterwards the researched techs get updated when ServerApp::PostCombatProcessTurns calls empire->CheckResearchProgress in ServerApp.cpp .

Afterwards only meter effects are applied and visibility recalculated. So the detection meter does change the same turn the tech gets researched, but TurnTechResearched can only see the tech from the next turn on.

Is that right? Can/should do something about this?
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: TurnTechResearched not working the first turn

#2 Post by Geoff the Medio »

Please clarify what you mean by "work for the turn in which the tech gets researched". ie. describe a scenario / script that you expect to do one thing that does another.

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#3 Post by Ophiuchus »

The original problem is coming from SitRep for enemy sensor tech advances? and suggests if the general underlying problem of effect application order cant be solved that it should at least be possible to hook sitrep messages into meter effects - so that a meter change gets correctly reported when it happens.
Geoff the Medio wrote: Fri Jan 04, 2019 4:53 am Please clarify what you mean by "work for the turn in which the tech gets researched". ie. describe a scenario / script that you expect to do one thing that does another.
Lets say you research a SPY_DETECT_2 detection tech. Research finishes on turn 20, you get the sitrep the tech was researched and the detection boost applies.

Before turn 20: TurnTechResearched for SPY_DETECT_2 returns 0; expected an indication that the tech is not researched, maybe that should be the max number instead of zero :?

turn 20: TurnTechResearched for SPY_DETECT_2 returns 0; expected turn 20 :evil:

turn 21 and following: TurnTechResearched for SPY_DETECT_2 returns 20; expected turn 20 :mrgreen:


Now I also checked OwnerHasTech behaviour: it would match starting turn 21 in the example, not on turn 20. But maybe in some case on would want a OwnerHadTech behaviour. I would rather have OwnerHasTech match in turn 20 and simulate OwnerHadTech behaviour where necessary using TurnTechResearched.

edit1: link originating problem and suggest sitrep activation for meters

edit 2: note that OwnerHasTech consistently has the same behaviour

edit 3: "in turn" -> "on turn"
Last edited by Ophiuchus on Wed Jan 09, 2019 4:03 pm, edited 1 time in total.
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#4 Post by Ophiuchus »

I see multiple ways to attack the underlying issue, depending what is seen as the problem.

I think there are some assumptions which are reflected in the code as is
  1. assumption: most effects should happen before creation of ships/buildings/techs, so the results of the next turn are more predictable
  2. assumption: meter effects need to apply to newly created ships/buildings/techs
  3. there is no greater design for effect application, but the code is like it is because of finding failures and fixing it (oh, at that stage the ships should be detected, so add another visibility calculation)
In my opinion the implementation leads to some confusion:
  1. The behavior is not documented and confusing to FOCS scripters (some effects in the same activation condition/EffectsGroup may trigger for newly created, some may not)
  2. Workarounds (e.g. manually delaying the meter changes in a researched tech so a sitrep may coincide) lead to inconsistent application of effects (e.g. some effects are manually delayed, some are not) for technical reasons. This may create confusion for players.

So some ways one could address these problems
  1. document the (inconsistent) behaviour for players and scripters
  2. create more consistent behaviour-especially for players-by delaying all tech meter effects by one turn using FOCS
  3. Base effect application only on priority, so there would be a e.g. COMBAT_PRIORITY where combat happens, PRODUCTION_PRIORITY in which buildings and ships get build etc. This would lead to a lot more transparency and control for scripters and hopefully simplify the implementation.
  4. differentiate between initialisation and meter effects. E.g. only apply the last meter adjustment to newly created buildings and ships, not to techs and not to existing buildings/ships. Or introduce in FOCS 'initialisationEffects =' which defaults for buildings and ships to the normal 'effects =' and for techs to no effects and may be changed for individual content; as a scripter I would put the detection tech upgrade sitrep along meter changes into the 'intialisationEffects' and have the same meter changes in 'effects'. This would lead to more consistency and control.
In my opinion solutions should address these problems by increasing simplicity, consistency, control, transparency (in that order).
So i think-if it is doable-unifying the priority system (solution nr 3) would be the best solution. I have the feeling it could help a lot of other cases where you want to achieve a certain effect in FOCS.

More manageable and pragmatic would be the explicit intialisation (solution nr 4) and i think it would lead to a good solution.

Whatever we do, we should extend the effect application documentation in the wiki.
Input greatly appreciated!
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13587
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: TurnTechResearched not working the first turn

#5 Post by Geoff the Medio »

Ophiuchus wrote: Fri Jan 04, 2019 10:46 amResearch finishes in turn 20, you get the sitrep the tech was researched and the detection boost applies.
The meaning of this is unclear; specifically, what does "in turn 20" mean? Is it different from "on turn 20"?

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#6 Post by Ophiuchus »

Geoff the Medio wrote: Wed Jan 09, 2019 1:53 pm
Ophiuchus wrote: Fri Jan 04, 2019 10:46 amResearch finishes in turn 20, you get the sitrep the tech was researched and the detection boost applies.
The meaning of this is unclear; specifically, what does "in turn 20" mean? Is it different from "on turn 20"?
I don't know if it is different because i am not english native, this might be a germanism. So "on turn 20" if you prefer.

Even if i messed up the words, i'd think the explanation was clear:
What i meant was the turn where
you get the sitrep the tech was researched and the detection boost applies
, when the user gets the "researched" message and the detection boost.
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Telos
Space Squid
Posts: 50
Joined: Thu May 09, 2013 4:46 am

Re: TurnTechResearched not working the first turn

#7 Post by Telos »

Edit: Greyed out parts I no longer believe are true.

Suppose it is currently turn 20. My opponent just got a sit-rep telling them that Active Radar has been researched. I can see on my screen that their detection circle got larger, and I can look in the Empire statistics table and see that their detection strength has gone up to 30. But, apparently scripts that ask for TurnTechResearched for that enemy and ActiveRadar still return turn 0 this turn, rather than turn 20, even though it's clear that the answer already should be turn 20, and indeed the answer of 20 will be returned by such scripts in all later turns. Similarly, scripts that ask whether OwnerHasTech for that enemy and that tech get the answer no this turn, even though it's clear from other aspects of the display that the answer should be yes. This is immediately relevant because it would be nice to have a script generate a SitRep for me about their detection tech advance *this*turn* so I can move my stealthy ships to safety before they get killed.

As a (temporary) work-around, can you make the SitRep be generated by the very same script (DETECT_2.focs) that adds the bonus distance to detection ranges and bonus detection strength to the empire? This script checks whether the empire has researched *later* detection techs, but it doesn't actually check whether OwnerHasTech for that tech itself (instead it's triggered by their actually having the tech), so this won't be thrown off by OwnerHasTech wrongly answering "no" for a turn. In fact, so long as it gives a wrong answer, you can use this as your indicator that this was the very turn that the tech was researched. And you can generate sitreps for all players at once by not specifying a particular empire to receive the sitrep, e.g. following the model in ANCIENT_RUINS.focs, though I fear that will have the effect of also sending "rumors" of the tech advance to the player who made it as well, so I suppose the SitRep would need to be phrased in a way that is neutral as to who will receive it, e.g., something like "The galaxy is buzzing with rumors that the [EMPIRE] has made advances in sensory detection abilities. Smugglers and pirates are shifting their routes accordingly."

Here's a very rough stab at that script, to go in DETECT_2.focs (and similar ones should go in DETECT_3, DETECT_4, and DETECT_5):

Code: Select all

        EffectsGroup
            scope = Source
            // This scripted will be executed only when the player has actually researched this detection tech
            // and (due to a bug) the following activation condition will still be met on the turn in which they just researched it, but not later.
            // (In case they fix this bug, it might be better to write a more complicated condition that will return the right answer post-fix too.)
            // Together those two facts ensure that this sitrep will get sent out to all players only the turn the tech is researched.
            activation = Not OwnerHasTech name = "SPY_DETECT_2"
            effects = [
                GenerateSitRepMessage
                    message = "EFFECT_DETECTION_ADVANCE_RUMORS"
                    label = "EFFECT_DETECTION_ADVANCE_RUMORS_LABEL"
                    icon = "icons/tech/active_radar.png"
                    parameters = [
                        tag = "empire" data = Source.Owner
                    ]
            ]
Of course you'll also need to edit your String Table (e.g., for English, en.txt) with something like the following.

Code: Select all

EFFECT_DETECTION_ADVANCE_RUMORS
The galaxy is buzzing with rumors that the %empire% has made advances in sensory detection abilities.  Smugglers and pirates are shifting their routes accordingly.

EFFECT_DETECTION_ADVANCE_RUMORS_LABEL
Smugglers Beware!
Last edited by Telos on Sun Jan 13, 2019 1:53 am, edited 2 times in total.

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#8 Post by Ophiuchus »

Telos wrote: Sat Jan 12, 2019 5:43 am This is immediately relevant because it would be nice to have a script generate a SitRep for me about their detection tech advance *this*turn* so I can move my stealthy ships to safety before they get killed.
I am not sure i read you correctly, but correct me: You would like to get a sitrep one turn before the detection boost applies?
I think we should keep the danger of being detected real ("surprise - we can see you, you are dead"). But I think a variant of this might be a good idea: allow the empire the possibility of smaller detection steps. E.g. instead of just 10,30,50 add DETECT_1B with 20, DETECT_2B with 40.. and research tree allows to "skip" the B tech to make detection change more surprising. Researching the intermediate tech DETECT_1B and DETECT_2 should be a cheaper than not researching DETECT_1B.
Telos wrote: Sat Jan 12, 2019 5:43 amAs a (temporary) work-around, can you make the SitRep be generated by the very same script (DETECT_2.focs) that adds the bonus distance to detection ranges and bonus detection strength to the empire?...
Activation condition is fine, does not need to be corrected. The problem is not in FOCS. With the current cpp code it is impossible to send a sitrep about a newly researched tech. Only those effects which affect meters are applied in that turn.
There is a workaround which delays the application of meter effects for one turn (see the other thread) in the FOCS code.
Telos wrote: Sat Jan 12, 2019 5:43 amHere's a very rough stab at that script, to go in DETECT_2.focs (and similar ones should go in DETECT_3, DETECT_4, and DETECT_5): ...
Did you actually try out your script in the game to see if it works?
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Telos
Space Squid
Posts: 50
Joined: Thu May 09, 2013 4:46 am

Re: TurnTechResearched not working the first turn

#9 Post by Telos »

I'd like to get the SitRep as soon as I could otherwise know that they've researched the tech, i.e., the same turn that (if I paid attention) I could see that their detection circles got bigger, and the same turn that their detection level gets higher in the Empires data table. The way timing in the game works, this is slightly after the tech comes into effect, because the enemy already has just gotten vision of things in the enlarged area, and of things that previously had been too stealthy for them to see. But this is still prior to the first *combats* in which the tech is relevant, so I still have the chance to give my stealth ships orders to move away (though if they're in a system that the enemy had blockaded before my arrival, then my movement options will be severely limited). So I guess the answer to your question is "half no" (I don't want the sitrep until just after the tech already took effect by enlarging their circles), but also "half yes" (I do want the sitrep before the first *combats* in which that tech will have effect).

EDIT: Greyed out the parts below that I no longer believe are true.

I agree with you that there is a problem in cpp code, making it give the wrong answers for TurnTechResearched and OwnerHasTech on the turn a tech is researched (answering "turn 0" and "no", when it should instead answer the current turn and "yes"). The solution I proposed does not require that either of these give the right answers (and in fact relies on their giving the wrong answers). We know the Detect_2 script *does* get executed on the turn the tech is researched because this script is what expands the sensor circles and makes previously stealthy things become visible that turn. So I'm just adding in there that it should also send out a SitRep to everyone the first time it does this.

No, I hadn't tried it yet, as I figured it would probably be easier for you as you probably had a test scenario set up to make it easy for you to generate a scenario in which the message should be triggered. But I think that (up to dumb mistakes, which I often make) it probably *should* work. If I get a chance, I'll try it later today.

(I think we're diagnosing the cpp problem differently. I think the problem is just that, whatever information TurnTechResearched and OwnerHasTech consult to provide their answers is slightly out of date, so they don't start giving the right answers until one turn late. When you use these in your activation condition, there's a sense in which your activation condition is "fine" (it would work fine if the cpp code worked in the best way), but another sense in which your activation condition is *wrong* (it doesn't currently work, whereas the one I proposed I think will). You seem to think that the problem instead has something to do with the server hiding information about tech advances from the client, and/or something to do with the game implementing only meter-affecting effects of a tech on the turn it is researched. I don't think this information can be all that hidden from the client, as it does show up on the client's display in other ways (larger detection circles, higher detection strength in the empires table, probably shows up as having been researched on the pedia screen for that empire -- that'd be interesting to check). And I don't know of any reason to think that the game does or should execute only meter-adjusting effects in the way you suggest, rather than executing all effects, so I doubt that really is the problem.)
Last edited by Telos on Sun Jan 13, 2019 1:51 am, edited 1 time in total.

Telos
Space Squid
Posts: 50
Joined: Thu May 09, 2013 4:46 am

Re: TurnTechResearched not working the first turn

#10 Post by Telos »

So I did indeed make one dumb mistake: putting the icon before the parameters in the SitRep generation call. Edited in the fix above so now it gets read in fine. Family duties demand my attention now, so will test it later.

Telos
Space Squid
Posts: 50
Joined: Thu May 09, 2013 4:46 am

Re: TurnTechResearched not working the first turn

#11 Post by Telos »

Ok, so I've now experimented quite a lot with this, and it turns out I wasn't entirely right, but I don't think the problem really has anything to do with TurnTechResearched nor OwnerHasTech.

Instead, I think the problem is just that I can't find an opportunity to execute scripts between the time when a new tech is researched (e.g., after everybody has hit End Turn at the end of Turn 19), and the time when the next available batch of SitReps are generated (e.g., the SitReps that will be visible on Turn 20, including the automatically generated SitRep indicating the completion of the research you expected to have completed). At any rate, when I add an effect into the tech itself to *always* spam players with SitReps, then the first such SitReps will appear the turn *after* the turn in which the Research Completed SitRep appears. And if I script that SitRep spam to stop after a particular turn, (say turn 30), then the last received SitRep will appear (on Turn 31) the turn *after* the last one was supposed to have been generated. So apparently, we should think of SitReps, or at least the SitReps generated by effects in Techs, as being slow-moving messages in bottles that won't arrive until the turn *after* the turn in which a script generated them.

Is there a location where we can place scripts so that they will execute early enough in the turn to generate a SitRep that will be readable *that*very*same* turn?

Note: this issue doesn't arise just for wanting SitReps to warn us about enemy sensor tech advances. It will also affect other SitReps that should be generated as soon as someone researches a new tech, like the helpful "nag" message suggesting "prime locations" for gas-giant generators, which it could be helpful to have be readable on the very turn that Orbital Generation is researched, but currently are not readable until the turn *after* that.

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#12 Post by Ophiuchus »

Telos wrote: Sun Jan 13, 2019 1:47 am..this issue doesn't arise just for wanting SitReps to warn us about enemy sensor tech advances. It will also affect other SitReps that should be generated as soon as someone researches a new tech, like the helpful "nag" message suggesting "prime locations" for gas-giant generators, which it could be helpful to have be readable on the very turn that Orbital Generation is researched, but currently are not readable until the turn *after* that.
If you look at the cpp code you will see like i said in the original post:
that normal effects happen and afterwards the researched techs get updated when ServerApp::PostCombatProcessTurns calls empire->CheckResearchProgress in ServerApp.cpp .

Afterwards only meter effects are applied and visibility recalculated. So the detection meter does change the same turn the tech gets researched, but TurnTechResearched can only see the tech from the next turn on.
See ServerApp.cpp starting directly after production and research progress (line 3326) to the call of m_universe.ApplyMeterEffectsAndUpdateMeters in line 3336).
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Telos
Space Squid
Posts: 50
Joined: Thu May 09, 2013 4:46 am

Re: TurnTechResearched not working the first turn

#13 Post by Telos »

So, here's how I understand what's going on, sticking with the example where the player gets their own research-complete SitRep on Turn 20.
  1. On Turn 19, the research screen shows that I'm 1 turn away from finishing research on Active Radar.
  2. I click to end my turn.
  3. The game processes various events, including ship movement and combat.
  4. The game then processes research, which completes the tech Active Radar, and automatically triggers the 3 SitReps I will see on the next turn telling me I've completed this research, and that a new building and shippart are now available.
  5. Some FOCS scripts are executed to adjust meters near the end of Turn 19, but, as far as I can tell, this does not yet include the scripted events for Active Radar, or if it does, those are not yet allowed to generate SitReps.
  6. Turn 20 officially begins.
  7. A list of SitReps is displayed, including the automatically generated ones produced by any completed research, but this will *not* include any SitReps generated by FOCS scripts in the techs that were just researched.
  8. At some point, during turn 20 FOCS scripts for researched techs are called, including newly researched techs like Active Radar. Any SitReps generated by these scripts, including ones that include CurrentTurn <= 20 as a condition, will not be visible on Turn 20, but will become visible on Turn 21.
  9. I click to end the turn again.
  10. Turn 21 officially begins.
  11. The first SitRep generated by scripts in the Active Radar tech finally appears.
This is an unfortunate sequence of events, as it precludes making scripts in a tech that will generate SitReps that will appear at the same time as when the SitRep telling the researcher they finished the tech appear, and at the same time as the first visible effects of that tech appear (like expanded sensor circles or being able to begin constructing buildings unlocked by the tech). This means that we can't have the Orbital Generation tech script "nag" SitReps that will appear on the turn Gas Giant Generators can first be built -- instead those messages will appear the *next* turn, after you've already had to manually search through your empire to find your Gas Giants. Similarly, the Active Radar tech can't script messages to immediately warn other players that an enemy's sensor circles just expanded -- instead those messages won't appear until the *next* turn, which puts stealth ships needlessly at risk.

This unfortunate sequence of events need not involve any calls to TurnTechResearched nor OwnerHasTech, so the problem is not (just) that those sometimes return the wrong results.

Instead, the problem appears to be that we can't script effects in techs that will be executed early enough to generate SitReps that will be immediately visible when the tech is first researched. One good solution would be to allow such scripts to execute right when the tech research is completed (e.g., near the end of turn 19) and generate SitReps to be visible at the same time as the SitRep indicating that the tech has been researched (on turn 20). If such scripts were possible, then you're right that it would be helpful to make sure that they get sensible results from TurnTechResearched nor OwnerHasTech, but since they aren't even possible at all right now, it doesn't much matter whether they would get sensible results from those or not. Another good solution would be to allow these scripts to be executed early in Turn 20 but still to generate SitReps that will be visible already on Turn 20, rather than having their visibility be delayed until Turn 21.

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#14 Post by Ophiuchus »

Just stumbled about an older discussion where The Silent One had problems with unintended meter effect application. Dilvish gave another overview of effect application.

Code: Select all

fleets_move
combat
all effects applied, including building new "current" meter values from "initial" meter values
supply & resources updated
things made
scripted events processed
meter effects applied again (rebuilds new "current" meter values from "initial" meter values, not adding another dollop to current meter values)
population growth
back-propagation of meters ("initial" values overwritten by "current" values)
turn number advanced 
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Ophiuchus
Programmer
Posts: 3433
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: TurnTechResearched not working the first turn

#15 Post by Ophiuchus »

An update, I made a PR#2370 in which the addition of techs is shifted to the end of turn processing which simplifies the turn processing.

In scripting terms it means you can use any effect you like in the same turn as the tech influences meter values.

In play terms this means that meter values will be affected starting with the turn after you research the tech (so one turn later than currently).
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Look, ma... four combat bouts!

Post Reply