FreeOrion

Forums for the FreeOrion project
It is currently Sun Oct 22, 2017 6:18 am

All times are UTC




Post new topic Reply to topic  [ 10 posts ] 
Author Message
PostPosted: Tue Dec 27, 2016 12:26 am 
Offline
Programmer

Joined: Sun Feb 14, 2016 12:08 am
Posts: 333
Recently noticed that boost::hash is not consistent across boost versions when resolving for the galaxy seed, at least on my system.
Sampling from the seed hash done in ServerApp, the result is different between 1.60 and 1.62
Though I doubt so, I am curious is if is portable (wrt OS) for the same boost version.

With the move to c++11, this could be changed to std::hash, which would at least reduce the case for boost version.
I do not see any guarantee of portable results from std::hash however, the implication is the opposite:
http://en.cppreference.com/w/cpp/utility/hash wrote:
The actual hash functions are implementation-dependent and are not required to fulfill any other quality criteria except those specified above.

If std::hash is not consistent across OSes, but boost::hash is (and the opposite case between boost versions), between the two I'd opt to keep boost::hash.

For the PRNG itself, std::mt19937 is "required" to produce the same results, but AFAIK none of the distributions like std::normal_distribution provide such requirement (same case for boost).
'Simply' providing a custom, portable value for seed translation might still provide different results between platforms without a custom distribution as well.

I'll open a PR for result requests from this branch, if it looks like std::hash may be desired.
(Previous results, so I don't lose them in such a case)
seed: aaaaBBBB = (std)2783040585 (boost 1_60)2892594671
seed: aaaaBBBB = (std)2783040585 (boost 1_62)1426502312

seed: aaaabbbb = (std)4214164133 (boost 1_60)3005869415
seed: aaaabbbb = (std)4214164133 (boost 1_62)2207442920

seed: 6uTwEu5q = (std)4078502888 (boost 1_60)4027358770
seed: 6uTwEu5q = (std)4078502888 (boost 1_62)707295000

Another option might be to only store the string value and push all the related PRNG calls to python, though I have not found as much discussion on portability there.

_________________
Any content posted should be considered licensed GNU GPL 2.0 and/or CC-BY-SA 3.0 as appropriate.


Top
 Profile  
 
PostPosted: Tue Dec 27, 2016 8:58 pm 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4226
Location: Sol III
That again... *sigh*. We had the issue of the same seed not producing the same galaxy on different platforms, but got that solved somehow - IIRC by doing all the random stuff in the Python scripts. Apparently the standard PRNG Python uses does produce the same results for the same seed across platforms.

AFAIK that relies on boost::hash to produce the same hash from the string seed on all platforms, and independent of the boost version. If there are now differences between boost versions, that's not good. As the primary concern here is to get consistent results during universe generation, all we need is a hash function that reliably produces the same hash from the seed string across platforms. If the hash functions provided by boost or the standard library fail to do so, how difficult would it be to implement our own hash function? It doesn't need to meet any sophisticated standards after all...


Top
 Profile  
 
PostPosted: Tue Dec 27, 2016 11:00 pm 
Offline
AI Contributor

Joined: Tue Feb 17, 2015 11:54 am
Posts: 222
If the hashing function makes trouble, why not get rid of the hashing function: Let's just use an integer seed directly.

_________________
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 12:32 am 
Offline
Creative Contributor
User avatar

Joined: Fri Jun 28, 2013 11:45 pm
Posts: 3245
I too thought we'd closed it, but if it's Boost dependent that would explain some variation.

I'm in the early stages of a restarted game and while layout, species, etc are all identical placement of guard sentinels/ancient guardians/monsters aren't, there was a vacuum dragon causing havoc in the first game, that's gone, and two planets that had guardians now have maintenance ships, another has nothing, etc.

If it's going to be worked on again, getting it so it's properly identical would be an objective I'd hope.

_________________
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 12:42 am 
Offline
Programmer

Joined: Sun Feb 14, 2016 12:08 am
Posts: 333
I've added a fairly simple hash function to that branch.
Not sure how I feel about reverting the string back to an integer wrt user interaction/display.

I suppose this means some python function/wrapper needs to be provided to replace RandSmallInt (and friends), at least where a portable value is desired?

I expect there is an additional issue for effects in content definitions, otherwise they would not show different behavior on the same system/setup.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 9:54 am 
Offline
Programmer
User avatar

Joined: Fri Mar 01, 2013 9:52 am
Posts: 1040
Location: Germany
Quote:
Recently noticed that boost::hash is not consistent across boost versions when resolving for the galaxy seed, at least on my system.
Sampling from the seed hash done in ServerApp, the result is different between 1.60 and 1.62
Though I doubt so, I am curious is if is portable (wrt OS) for the same boost version.


You're running a 64 bit OS, right?

https://github.com/boostorg/functional/ ... a00efdbeb5

_________________
Resident code gremlin
Attached patches are released under GPL 2.0 or later.
Git author: Marcel Metz


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 10:43 am 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12004
Location: Munich
dbenage-cx wrote:
I expect there is an additional issue for effects in content definitions, otherwise they would not show different behavior on the same system/setup.
As noted elsewhere, multithreading affects the results of a/the stateful PRNG used for resolving randomness in effects evaluation. This applies directly, through explicitly multithreading of the effects code, and probably indirectly, due to AI clients requesting object ids from the server, with the results depending on the order the requests are received, and the numbering of objects affecting how they're stored in containers indexed by object id, which can in turn affect what is selected randomly from those containers or the order in which effects are evaluated.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 11:33 am 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4226
Location: Sol III
dbenage-cx wrote:
I suppose this means some python function/wrapper needs to be provided to replace RandSmallInt (and friends), at least where a portable value is desired?
Not really. Just checked the universe generation scripts, and they don't use the hash produced on the C++ side at all. The seed string is passed to Python as the player entered it, and there the hash value is determined by using the MD5 hash function provided by the Python standard library. So on the Python side we are already completely independent of hash functions provided by boost or the C standard lib and all the hassle that comes with them.

Which means the problem actually exists only on the C++ side of things, and for the reasons Geoff pointed out I don't see how we could achieve reproducable sequences of random values anyway.

Concerning the placement of guard monsters etc. the only solution I see is to take that out of the effectsgroups of the relevant specials and migrate it to the universe generation scripts. As well as everything else that actually is part of universe generation and therefore should produce the same results for the same seed across all platforms.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 11:42 am 
Offline
Programming, Design, Admin
User avatar

Joined: Wed Oct 08, 2003 1:33 am
Posts: 12004
Location: Munich
Vezzra wrote:
Which means the problem actually exists only on the C++ side of things, and for the reasons Geoff pointed out I don't see how we could achieve reproducable sequences of random values anyway.

Concerning the placement of guard monsters etc. the only solution I see is to take that out of the effectsgroups of the relevant specials and migrate it to the universe generation scripts. As well as everything else that actually is part of universe generation and therefore should produce the same results for the same seed across all platforms.
For universe generation purposes, it could probably be set up to use only a single thread while evaluating those start-of-game effects. There's no AI-client timing during that stage either.


Top
 Profile  
 
PostPosted: Wed Dec 28, 2016 11:52 am 
Offline
Release Manager, Design
User avatar

Joined: Wed Nov 16, 2011 12:56 pm
Posts: 4226
Location: Sol III
Geoff the Medio wrote:
For universe generation purposes, it could probably be set up to use only a single thread while evaluating those start-of-game effects. There's no AI-client timing during that stage either.
If we manage to solve the issues with the hash function and PRNG on the C++ side as well, then yes, that could be possible, at least for effects processing on turn 1. But that feels a bit like a hack. IMO moving everything universe generation related into the universe generation scripts sounds like a reasonable approach anyway, so why not take that road?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group