Scripted Universe Generation!

Programmers discuss here anything related to FreeOrion programming. Primarily for the developers to discuss.

Moderator: Committer

Message
Author
User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#46 Post by Vezzra »

MatGB wrote:On the original thing, broadly what do you need to amend/do to make your own layout? I keep looking at the code and my eyes glaze over but I'm sure once I get the basic principle it'll be easy to go forward with the rest.
You mean some introduction/tutorial/manual/ on how to customize the universe generation script/make your own one? I've definitely planned that, it's just that this is a quite time consuming task. Add to that the fact that scripted universe generation is still only partly done, there are still major steps/things that haven't been ported to Python yet (generation of natives, specials, individual starlanes etc.). So if I made even just a brief introduction, I'd had to take that into account and give proper explanations what can and can't be done, which have to be adjusted accordingly with each thing that gets ported to Python. That's a bit much, I only have so much spare time I can allocate to FO development, so I thought I postpone that at least until scripted universe generation is a bit more complete.

However, if there is already real demand for that, I can of course do it. One thing to keep in mind although is, you need to dig into the Python programming language, that is, you need at least to learn some basics about it (and programming in general of course, for those who haven't done anything in that regard yet), I don't think I'll be able to simplify things so much that tweaking the universe generation scripts is in the same league as the other content scripts. So, until I manage to put up some kind of universe-generation-script-howto, you can head over to http://docs.python.org/2.7 and enjoy the tutorial ;) (FYI, FO uses Python 2.7)

User avatar
eleazar
Design & Graphics Lead Emeritus
Posts: 3858
Joined: Sat Sep 23, 2006 7:09 pm
Location: USA — midwest

Re: Scripted Universe Generation!

#47 Post by eleazar »

Now that we have star group naming perhaps we should consider making a cluster placement visually distinct. I'm going to use the term "cluster". And of course "stars" refers to black holes and deep space too. This would make clusters more of a meaningful geographic feature rather than just a naming convention. As part of that it could be interesting if stars in a cluster had a higher probability of starlanes between the members.

I.E.
  • Stars in a cluster would generate at maximum N UUs from another members of the cluster.
    Non-cluster stars would generate at minimum N +X UUs from any other stars.
    X would be enough UUs to make clusters visibly closer than regular stars.
    And of course stars in a cluster should still have a sensible minimum allowed distance.
The prevalence of clusters could plausibly be a universe gen option, but since part of the point of clusters is to make more names for very large galaxies, above certain sizes low cluster settings would have to be disallowed, or ignored.

User avatar
cami
Space Dragon
Posts: 411
Joined: Tue Sep 20, 2011 10:32 pm
Location: Halle (Saale)

Re: Scripted Universe Generation!

#48 Post by cami »

This is an excellent idea, considering the tactical consequences. And it was born out of a discussion about naming conventions! Who said discussing naming is weird? It's fruitful for tactical gameplay :p

Warning: this post may be pointless. By reading it, you agreed to waive any entitlement to compensation for your wasted time.
Yesterday, we were still on the brink. Fortunately, today we have come one step further.

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#49 Post by Vezzra »

This are some comments in response to Dilvish and his starname patch v3 which I said I'm going to post here because they are more concerned with the universe generation script in general than with star system naming in particular.

Ok, Dilvish, here are my thoughts/comments that came up when I took a look at v3 of your patch (and also when I read some of Mats comments on my Python code). These have been in the back of my mind already, but I put them off because I first wanted to complete scripted universe generation at least to the point where all major parts have been ported to Python, and the original behaviour could be reproduced with only limited use of legacy universe generation code.

But as a first major patch to the universe generation script is already about to go in, I think I better bring these thoughts and comments up now. It concerns structure and coding style of the script.

What I'm aiming for is that even people (content scripters) who do not have very much experience with programming have a decent chance of understanding and being able to tinker with the universe generation script. Of course I'm aware that at least some basic experience with programming and knowledge/familiarity with Python is going to be required, it won't be possible to simplify things to the point where one could customize it without that. But I'd want to try to get as close as possible.

To achieve that, I think there are several things we need to take care of when it comes to the structure and code style used in the universe generation script:

1.) Code readability: We need to strive for simple and readable code and be willing to sacrifice efficiency, coding elegance and performance for that (wherever possible of course). I've already had second thoughts about my Python code and wondered how I could make it cleaner, simpler, more readable, because I've been worried that someone with only basic experience with programming might not get past "Huh?" when looking at my code. A suspicion that Mat has confirmed (thanks, Mat, btw, I know that probably wasn't your intention, but it gave me some very valuable feedback ;)).

Now, I think I already have a rather simplistic code style, as my own Python skill are rather basic, and I don't know about most of the more sophisticated/fancy features of the Python language. Contrary to you, you're obviously much more adept at Python (and programming) than I am, and I think your coding style reflects that. IMO your code is more compact, efficient and makes more elegant use of the various features a programming language offers. But it also makes it harder for someone not so experienced to understand what's going on. To illustrate what I mean I'll give an example taken from you starname patch:

Code: Select all

    stargroup_words[:] = getNameList("STAR_GROUP_WORDS")
    stargroup_chars[:] = getNameList("STAR_GROUP_CHARS")
    stargroup_modifiers[:] = [stargroup_words, stargroup_chars][ star_groups_use_chars ]
Maybe I'm wrong, but if a understand that piece of code correctly, the following should be equivalent, at least for the purposes of your patch:

Code: Select all

    stargroup_words[:] = getNameList("STAR_GROUP_WORDS")
    stargroup_chars[:] = getNameList("STAR_GROUP_CHARS")
    if star_groups_use_chars:
        stargroup_modifiers[:] = stargroup_chars
    else:
        stargroup_modifiers[:] = stargroup_words
While your version is certainly more compact (I mean, one line with a simple assignment instead of a four line if...else construct), I think my alternative is definitely more readable, especially for someone with only limited experience with programming/Python. Such a person will probably look at that line and wonder what you're doing here and why.

2.) Encapsulation: Of course there are things the universe generation scripts needs to do that simply can't be done simple and readable. Some, because the performace hit would be too bad, others won't be doable that way at all. These parts should be encapsulated in a set of functions (maybe classes) with an interface as clean and simple as possible, that (universe generation) content scripters can understand and use without too much difficulty.

IMO that of course requires that the whole process of universe generation should be split up into distinct steps with as less intertwining as possible. Again, taking your starname patch as example: I've seen you adjusted some of the star system generation code, because you already need to determine the star type of a system before it gets actually created for your name generation code to work. And you don't create a system before determining its name because the system generation API function call already expects a name. OTOH, now these two steps (star system generation and name assignment) are intertwined, making the whole code section more complicated and less understandable for the unexperienced programmer.

In this particular case my suggestion would be to "disentangle" these two steps. First, create all the star systems and it's contents (planets) as it has already been done before with temporary placeholder names (e.g. "System " + system id). Then, in a completely seperate step, the content scripter can invoke the whole naming process just by calling a function we provide for him, which encapsulates the entire complexity of that mechanism.

All that leads to my third point:

3.) Modularization: I've started off with just one Python file containing the entire script, but even at the very beginning I've been aware that most likely I won't be able to keep it that way. Even if universe generation isn't going to be anywhere near as extensive and complex as the AI scripts, I think we should split it up into several modules which get imported and employed by the main script (like it's done with the AI scripts). It has been already quite big even without your patch, but now I think we've reached the point where we really should do that. I suggest you commit your patch, then I'll make the necessary preparatory changes, that is, create a dedicated subfolder ("default/universe_generation" or something like that) and move the universe generation script there (I don't think cluttering the default folder with a load of universe generation Python script files is a particulary brilliant idea ;)).

Then we can start to break things up into modules. E.g. I'd move the whole star system naming thing in a separate module (maybe "starnames.py"?), and so on.

4.) Setting module: With modularization in place, I think it would be a good idea to put all "setting" variables like the ones you introduced to control certain aspects of starname generation in a dedicated module ("setting.py") (assuming there might be more of these settings variables to come). That way, we can point anyone who wants to tinker with them to a single location where everything is neatly put together and no other code around confusing people who don't have any clue about programming at all.

Ok, that's my ideas so far. What do you think?

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#50 Post by Vezzra »

eleazar wrote:Now that we have star group naming perhaps we should consider making a cluster placement visually distinct.
Hm, interesting idea, although that of course goes far beyond mere star system naming - that would be a significant modification to the layout of the galaxy maps. It would introduce some kind of clustering to all galaxy shapes, and I don't know how well that's going to work e.g. for the spiral ones. Will probably depend on how noticeable/marked (I hope I got the right words for what I want to say...) you want these "micro-clusters" to be.

I also wonder how these are going to work with the cluster galaxy shape. Clusters of clusters?
The prevalence of clusters could plausibly be a universe gen option,
That could work, and probably replace the cluster galaxy shape? That way you could make any galaxy shape "clustered", to reproduce the current cluster galaxy shape you could take irregular and set the number of stars per cluster to "high" (assuming "prevalence of clusters" = number of stars per cluster, with "none" meaning no clustering).

On a second thought, we may need a second option that controls the difference of the average distance between stars in a cluster versus the average distance to stars outside a cluster/belonging to another cluster. The bigger that difference, the more noticeable/marked the individual clusters will be.
but since part of the point of clusters is to make more names for very large galaxies, above certain sizes low cluster settings would have to be disallowed, or ignored.
Agreed... if the treshold is high enough. Number of "individual"/"solo" names in the star names list would be an obvious choice.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12745
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Scripted Universe Generation!

#51 Post by Geoff the Medio »

Could you move the UniverseGenerator.py file into a subdirectory, similar to AI? It, and the .pyc files generated, seem inappropriate in the root default directory...

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#52 Post by Vezzra »

Geoff the Medio wrote:Could you move the UniverseGenerator.py file into a subdirectory, similar to AI? It, and the .pyc files generated, seem inappropriate in the root default directory...
Yep, that's the plan. See point 3 in my post above.

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#53 Post by Vezzra »

It has taken me far longer than I thought, but I finally got around to implement the ideas outlined in my post above. I've moved the UniverseGenerator.py script to it's own subdirectory (actually this step already happened a while ago) and split it up into several modules. Doing that I also tried to encapsulate some processes, particularly the more complicated star group naming mechanism Dilvish implemented. I completely "disentangled" it from the actual star system creation and population process., it's now done after everything else has been completed. And finally I moved all settings/options variables (currently these are only the star group naming options) into it's own module (options.py).

Some brief testing indicates that I haven't broken anything, @Dilvish: if you want, you might check if I didn't mess up anything in your star group naming thing (has been moved to starnames.py). I've tried to minimize my tinkering there, but some small adjustments of course have been necessary. OTOH, you might want to wait a bit, because I'm planning to add starlane functions (both for manipulation and getting info about them) to the Python interface soon. I guess you may want to adjust the whole mechanic then to take starlane connections into consideration when determining clusters.

I think the whole script is far better organized and clearer now. Please, everybody who is going to tinker with these scripts, try to maintain a high level of organization, readability, encapsulation and modularization (like I outlined in my post above). The universe generation scripts need to be readable by content scripters without advanced programming skills and an intricate knowledge of all the fancy stuff you can do with Python ;)

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#54 Post by Vezzra »

Vezzra wrote:@Dilvish: if you want, you might check if I didn't mess up anything in your star group naming thing (has been moved to starnames.py). I've tried to minimize my tinkering there, but some small adjustments of course have been necessary. OTOH, you might want to wait a bit, because I'm planning to add starlane functions (both for manipulation and getting info about them) to the Python interface soon. I guess you may want to adjust the whole mechanic then to take starlane connections into consideration when determining clusters.
Ok, starlane adding, removing and querying has been added. It should now be possible, when determining star system groups, to take the starlane connections between them into consideration.

User avatar
Cjkjvfnby
AI Contributor
Posts: 480
Joined: Tue Jun 24, 2014 9:55 pm

Re: Scripted Universe Generation!

#55 Post by Cjkjvfnby »

1) I know you like patches, but first I want to show simple diffs, just to ensure it is ok for new coders. (still need to remove todo and make more testing)

remove function that can be onliner and used once
use zip to pair some values
https://github.com/Cjkjvfnby/freeorion/ ... 481419c38a

Reorganise some code in modules
https://github.com/Cjkjvfnby/freeorion/ ... 90a9871723


2) I prefer to import module items to scope not full module.

Code: Select all

import util
util.load_string_list(...)

from util import load_string_list
load_string_list(...)
3) You don't add new line at end of files.

4) Possible changes in generate_home_system_list https://github.com/Cjkjvfnby/freeorion/ ... a5c36fa8d6
Change inner loop to flat structure:
- select all good planets and distant. (if enough return)
- select all distant planets (if enough return)
- select any planets (if enough return)
PS. This code is not tested, I did not setup my pc for compiling.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
adrian_broher
Programmer
Posts: 1155
Joined: Fri Mar 01, 2013 9:52 am
Location: Germany

Re: Scripted Universe Generation!

#56 Post by adrian_broher »

Cjkjvfnby wrote:1) I know you like patches, but first I want to show simple diffs, just to ensure it is ok for new coders. (still need to remove todo and make more testing)

remove function that can be onliner and used once
use zip to pair some values
https://github.com/Cjkjvfnby/freeorion/ ... 481419c38a

Reorganise some code in modules
https://github.com/Cjkjvfnby/freeorion/ ... 90a9871723
Here, have some magic:

https://github.com/Cjkjvfnby/freeorion/ ... c38a.patch

https://github.com/Cjkjvfnby/freeorion/ ... 1723.patch
Resident code gremlin
Attached patches are released under GPL 2.0 or later.
Git author: Marcel Metz

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#57 Post by Vezzra »

Cjkjvfnby wrote:remove function that can be onliner and used once
use zip to pair some values
https://github.com/Cjkjvfnby/freeorion/ ... 481419c38a
These changes look good. To answer your question in the TODO: No to both. The order of the systems in home_systems is random anyway, so you can remove the reverse, and we also don't need to check if home_systems contains enough elements. If the function compiling this list can't find enough home systems, it throws an exception, and execution will be aborted before it reaches that loop. At least, my original function did, but your changed version doesn't seem to act much different from that (as far as I can tell).
If name_planets should be moved to the starnames module is probably a matter of taste. I actually intended to starnames module to contain all the logic for naming star systems specifically (because with all the star group naming stuff that's quite a lot of code, and I wanted to encapsulate that in a separate module).

Moving the call to name_planets into name_star_systems isn't something I'd want to do. I've intentionally kept them separate, because they might end up not being called together.
I prefer to import module items to scope not full module.
That, too, is a matter of personal taste. I prefer to import full modules (less headache regarding name conflicts). However, when using only one function (or very few) from a module, importing to scope probably makes sense.
You don't add new line at end of files.
Well, oops. I remove them if I notice them, obviously I missed some ;)
Possible changes in generate_home_system_list https://github.com/Cjkjvfnby/freeorion/ ... a5c36fa8d6
Change inner loop to flat structure:
- select all good planets and distant. (if enough return)
- select all distant planets (if enough return)
- select any planets (if enough return)
After a first look I think that should work, and your code is of course far more compact. However, I wonder if a scripter with only a basic understanding of programming will get past "Huh?" when he looks at that. Of course we can decide to provide this function to be used by scripters, they don't necessarily need to understand it's inner working. If they want to implement a different method of choosing home systems, they can simply discard our function and write their own. OTOH, I want scripters to be able to understand as much as possible of what the universe generation scripts are doing.

Anyway, in this case that's all a moot point, as this function needs to be rewritten. The current mechanic is problematic, because in larger galaxies with only a few empires the starting systems don't get distributed evenly enough. Some empires can end up quite close to one another, while leaving huge areas of the map without any empires within. We've to come up with some method to optimize that. Of course, if you have an idea how to do it, go ahead! :)
PS. This code is not tested, I did not setup my pc for compiling.
Well, grab the latest test build and replace the repective parts of the Python scripts with your code. That's an important point after all for doing certain things with scripts instead of hardcoding them in C++. So you can tweak things without having to be able to compile :D

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: Scripted Universe Generation!

#58 Post by MatGB »

Yup, no compiling needed for python script changes, which is the only way I can contribute anything ;-)

Still reading this thread, but the details of the discussion currently are way over my head, if there's going to be another functional change I'll have opinions but...

One thing that I would like to see, at some point, "scenario" generation involving set planet names as well as system names, so if naming a planet is possible that'd be good, but it's a low priority compared to some of the other things I know are planned.
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

User avatar
Vezzra
Release Manager, Design
Posts: 5415
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Scripted Universe Generation!

#59 Post by Vezzra »

I've finally had enough of the current silly implementation for the "Python Test" galaxy shape and replaced it with something more interesting (I hope). Similar to "irregular", but a bit different. The difference is probably more noticeable with a high number of star systems. Check it out! ;)

User avatar
MatGB
Creative Contributor
Posts: 3310
Joined: Fri Jun 28, 2013 11:45 pm

Re: Scripted Universe Generation!

#60 Post by MatGB »

Cool, I shall do som, um, now, yeah, the game I started yesterday was dull anyway.
Mat Bowles

Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Post Reply