Effects: Describe/Implement as Python-scripts?

[Ellestar]

It may be almost as descriptive as XML to designers, but not to the encyclopedia and AI.

--

As i understand, Python will not check for bugs in effect description and a designer must be sure that an effect he writes will be compatible with a design document and other effects? Or maybe you want to make it so compatibility will be ensured by a system? I don't understand, how effects system on Python will work (not just one of them, but how all complex will work). Can you write a description from a programmer's perspective?

If all service code will be written on C++ (maybe including a part that makes python code compatible with a design document, all C++ functions you want to call from a Python), and only a part that is exposed to a designer is written on a Python, then you need something like GLE in any case - it will be a service code.
And there will be a very small difference between GLE as a service to a Python and GLE that is also processes XML, because you're using C++ functions, say, to set meter value. So, the difference is that with Python effect group logic is defined in a Python and you're calling C++ functions from Python. With XML you're "compiling" effect group in several C++ function calls that are almost the same functions as C++ functions that are used by Python.

But maybe i don't understand what you want.

[a_claudiu]

I'm an experienced programmer and at this moment I'm tired listening about using XML anywhere for everything. XML is nothing more than an ini file with tree structure and basic validation.
I will try to make some points:

Pro XML:
- internationalization is easier
- you have some editors for it
- is having basic validation
- you have basic support for versions of the document structure (the code for interpreting and converting the old version still need to be written)

Cons XML:
- XML is not human readable!!! Except the basic examples that you see in the books with 3 elements a real XML file is very hard to read.
- XML is not a programming language, is just a syntax and quite a strict one. Attempts of using it for describing programming languages have been made and I'm not very sure that are very succesfull (eg. XSLT)
- without an editor for an XML file forget about maintenability

My sugestion is to use it (if you really want) in describing static data like tech tree, some ini file containing color=red, x=30, but you must consider writing a special editor for those files (nice to have).

If you want to really mod the AI, the special events or some special conditions use a scripting language like Pyton. If you are afraid that can be too slow just make the modules separate and loaded dinamically (like dll's) with a specified interface than can be implemented in C++ or Pyton or both of them. This will keep open the oportunity in the future to change your mind and repair your potential mistake.

[Daveybaby]

^
|
|
Agree.

XML is just the latest thing - people seem to feel the need to use it for *everything* even when it doesnt need it. Been getting this at work a lot lately. Management decree that the project WILL use XML, leaving us poor code monkeys to try to shoe-horn it into a project which (a) doesnt need it, and (b) doesnt really suit it.

[LaplaceOperator]

Hey Yesterday i found the perfect alternative to this XML/phyton-stuff:
I think by having a look at this link you'll see, that there is no further need to continue this discussion because here i bring you the ultimate solution to all of fo's scripting needs :
http://shakespearelang.sourceforge.net/ ... peare.html

[Impaler]

Ophelia:
Thou art as disgusting as the quotient between Romeo and twice the
difference between a mistletoe and an oozing infected blister! Speak
your mind!

My god this stuff is hilarius, we should scrap all out code and start over from scratch with this stuff

As for this XML stuff I vote for what ever is most usable by the programing illiterate, their the ones who will be doing most of the Moding and Balancing of the game and the better they can access stuff the better. The Alpha file in SMAC was nearly perfect all it needed was to expose more of the games effects to modding.

[Yoghurt]

Code: Select all

[Enter Laser Mark II]

Ophelia: Thou art as tough as Armor Mark II, and two times as powerful as Laser Mark I. Thou requireth Quantum Field Dynamics III to be researched. Smash them down!

Hmm, yes. Could work!

[Vadus]

I guess that some other games were already made in Python. Maybe someone have an information, how fast they are (we need informaiton about games that are more complicated than tetris).

Maybe you heard, that Civ IV will use python scripts for all modable stuff, and Firaxis wouldn't programm with python, if it slows down the hole game.

And remember, that the current (and future) FO community will play (and mod) CivIV as well.

[Mikko Ohtamaa]

I am an experienced programming with a lot of background from different tools. I have seen games with modding and scripting options:

- Ghost Recon, XML based files
- Farcry, LUA scripting
- Quake, Custom QuakeC
- Doom, hardcoded switch loops
- Unreal, UnrealScript, JavaScript style
- Abuse, Lisp
- Alien vs. Predator, hardcoded C. One C source file (marine AI) was over 500 kb. C syntax is pretty heavy...
- Dukem Nukem 3d, pseudo C-like scripting language
- Vampire: The Masquare. If I recall correctly, used Java

Some opinions

1. Speed is not an issue. XML parsing is not faster than Python parsing. If something is hotspot, you can implement it in C++ later on.

2. When doing complex things, XML begings to suck. XML is good for structured information, not for event systems. For example, XML bends really badly to write a validation language in XML (XML Schema). And XML is not for manual typing, it's for machine to machine communication. I have been creating XSLT templates. Typing XML is soooooo slow...

3. Don't do logic in C++. You save a lot of manual typing, a lot of typing bugs, game is easier to extend etc.

4. Doing things in real scripting language (Python) instead of XML gives some real efficient boosters. For example, you can fill in table data in for loops.

5. Generate Python interfaces automatically from C++ headers. Use SWIG & etc. Interfaces are always synced up-to-date

6. Use Python. It's the best choice for a game engine from Python, Perl, Ruby, Java, Javascript and LISP (yuck..LISP!). Python is more flexible than Lua from syntax and extensibility viewpoints. Python integrates better with C++. Python execution overhead is not that big, especially if you compare to homebrewn solutions and XML.

7. Python integrates well with Boost. Boost is already used pretty much in FreeOrion.

8. Just do low level code (graphics integration, etc.) in C++. For high level stuff, use high level language. Every minute saved by using proper tools is a minute closer to 1.0 release. This is especially important if you have scarce human resources like open source projects usually do.

9. Python provides automatic serialization. No need to do manually write those kinky XMLEncodes...

10. Even Python has its quirks. But after you learn to live without typing ; after every sentence, you don't want to go to back to C++ or Java...

Some examples:

Code: Select all

<BuildingType>
  <name>BLD_MEGALITH</name>
  <description>BLD_MEGALITH_DESC</description>
  <build_cost>30</build_cost>
  <build_time>5</build_time>
  <maintenance_cost>20</maintenance_cost>
  <effects>
    <EffectsGroup>
      <scope>
        <Condition::Contains>
          <Condition::Self/>
        </Condition::Contains> 
      </scope>
      <activation>
        <Condition::MeterValue>
          <meter>METER_CONSTRUCTION</meter>
          <low>30</low>
          <high>999</high>
          <max_meter>0</max_meter>
        </Condition::MeterValue>     
      </activation>
      <stacking_group>MEGALITH_LOCAL_EFFECT</stacking_group>
      <effects>
        <Effect::SetMeter>
          <meter>METER_POPULATION</meter>
          <value>Target.MaxPopulation+1</value>
          <max>1</max>
        </Effect::SetMeter>
        <Effect::SetMeter>
          <meter>METER_TRADE</meter>
          <value>Target.MaxTrade+1</value>
          <max>1</max>
        </Effect::SetMeter>
      </effects>
    </EffectsGroup>
    <EffectsGroup>
      <scope>
        <Condition::And>
          <Condition::WithinStarlaneJumps>
            <jumps>1</jumps>
            <condition><Condition::Self/></condition>
          </Condition::WithinStarlaneJumps>
          <Condition::EmpireAffiliation>
            <empire_id>Source.Owner</empire_id>
            <affiliation>AFFIL_SELF</affiliation>
            <exclusive>1</exclusive>
          </Condition::EmpireAffiliation>
          <Condition::Type>OBJ_POP_CENTER</Condition::Type>
          <Condition::MeterValue>
            <meter>METER_CONSTRUCTION</meter>
            <low>30</low>
            <high>999</high>
            <max_meter>0</max_meter>
          </Condition::MeterValue>     
        </Condition::And>
      </scope>
      <stacking_group>MEGALITH_NEIGHBOUR_EFFECT</stacking_group>
      <effects>
        <Effect::SetMeter>
          <meter>METER_CONSTRUCTION</meter>
          <value>Target.MaxConstruction+3</value>
          <max>1</max>
        </Effect::SetMeter>
      </effects>
    </EffectsGroup>
  </effects>
  <graphic/>
</BuildingType>

could be expressed in Python, more compact and more readable:

Code: Select all

class Megalith(Building)

    def __init__(self):
        self.maintenance_cost = 20
        self.build_cost = 10
        self.time = 5
        self.name = "BLD_MEGALITH" 

        effects = {
		"MEGALITH_LOCAL_EFFECT" : self.doNeighbourEffect, 
		"MEGALITH_NEIGHBOUR_EFFECT" : self.doLocalEffect,
	}

   def doLocalEffects(self, local, neighbours):
	local.maxPopulation += 1
	local.maxTrade += 1       

   def doNeighbourEffect(self, local, neighbours):
       for neighbour in neighbours: 
	   if(distance(local, neighbour) < 1.0 and neighbour.owner == local.owner):
           	neighbour.maxConstruction += 3
    

And some C++ code could be replaced in Python (note that you don't need extra .h code too!):

Code: Select all

///////////////////////////////////////////////////////////
// SetPlanetType                                         //
///////////////////////////////////////////////////////////
SetPlanetType::SetPlanetType(const ValueRef::ValueRefBase<PlanetType>* type) :
    m_type(type)
{
}

SetPlanetType::SetPlanetType(const GG::XMLElement& elem)
{
    if (elem.Tag() != "Effect::SetPlanetType")
        throw std::invalid_argument("Attempted to construct a Effect::SetPlanetType from an XMLElement that had a tag other than \"Effect::SetPlanetType\"");

    m_type = ParseArithmeticExpression<PlanetType>(elem.Text());
}

SetPlanetType::~SetPlanetType()
{
    delete m_type;
}

void SetPlanetType::Execute(const UniverseObject* source, UniverseObject* target) const
{
    if (Planet* p = universe_object_cast<Planet*>(target)) {
        PlanetType type = m_type->Eval(source, target);
        p->SetType(type);
        if (type == PT_ASTEROIDS)
            p->SetSize(SZ_ASTEROIDS);
        else if (type == PT_GASGIANT)
            p->SetSize(SZ_GASGIANT);
        else if (p->Size() == SZ_ASTEROIDS)
            p->SetSize(SZ_TINY);
        else if (p->Size() == SZ_GASGIANT)
            p->SetSize(SZ_HUGE);
    }
}

std::string SetPlanetType::Description() const
{
    std::string value_str = ValueRef::ConstantExpr(m_type) ? UserString(lexical_cast<std::string>(m_type->Eval(0, 0))) : m_type->Description();
    return str(format(UserString("DESC_SET_PLANET_TYPE")) % value_str);
}

class SetPlanetType(Effect):

     def getDescription(self):
         return localize("DESC_SET_PLANET_TYPE")

     def apply(self, target, **kwargs):
         type = kwargs["type"]
         self.target.type = type
         if(type == "PT_ASTEROIDS"):
              self.target.size = "SZ_ASTEROIDS"
         else:
              self.target.size = "SZ_GAS_GIANT"

etc.

My recommendation:

1. Generate a framework which automatically generates Python headers from C++ headers. If you want to could even plug-in another scripting language (see Gimp).

2. Move all logic and data definition to unified Python format
- All game logic
- All event handling
- All game data definitions except localization strings
- Even UIs can be coded in Python
- Give up with XML/custom C++ serialization and use automatized serialization facilities

3. See your productivity getting 5x boost. Easy scripting language will also attract not-so-hardcore programmers for your programming team.

(Edited by Yoghurt: inserted code-tags)

[Sapphire Wyvern]

I'm not very familiar with Python or XML. But you may be interested to know how the Civ IV designers did it.

Seems that they went for a Python/XML based system - there's an explanation of why they chose those languages. Maybe their comments will add something to this discussion...

[Wolverine]

I'm great supporter of Python. It's easy, clean, integrates very well with C/C++ and Boost. Ogre3D and CrystalSpace 3d engines have support for Python. AFAIR Python's code embeed in C++ if properly coded can be almost 95% as fast as equivalent code in C++. Big pro for Python is that it's entirely object oriented. I've heard about few real time commercial games (even one commercial MMORPG, don't remember the name though) are using Python for all game logic. Also. Python objects can be easily serialised/deserialised and passed through unix/network sockets, CORBA, DCOP, XML and even as binary files. This could IMO allow for better networking control.

Python interpreter has quite big memory footprint (around 12MB on my machine), however it can be reduced by not including entire <python.h>, but rather only parts that are really needed (see header file).

And there's also other big pro. Psyco, that can be included for embeed C++. This is JIT compiler for Python that achieves significant speed improvements over repetetive functions (maths especially) that allows programs written in Python perform with speed comparable to C++, sometimes even slightly better. Some intensive scripts could be compiled with Psyco before using them in FO. I don't have any data regarding memory footprint though. And unfortunatelly I'm not aware of Psyco's portability issues, because I'm using Linux for development.

Guess this might shed some light on my opinion why Python is the way to go.

[Wolverine]

This is what I have found regarding Psyco (form psyco.sf.net). Hope you'll find it worth considering.

n short: run your existing Python software much faster, with no change in your source.

Think of Psyco as a kind of just-in-time (JIT) compiler, a little bit like what exists for other languages, that emit machine code on the fly instead of interpreting your Python program step by step. The difference with the traditional approach to JIT compilers is that Psyco writes several version of the same blocks (a block is a bit of a function), which are optimized by being specialized to some kinds of variables (a "kind" can mean a type, but it is more general). The result is that your unmodified Python programs run faster.

Benefits

2x to 100x speed-ups, typically 4x, with an unmodified Python interpreter and unmodified source code, just a dynamically loadable C extension module.

Drawbacks

Psyco currently uses a lot of memory. It only runs on Intel 386-compatible processors (under any OS) right now. There are some subtle semantic differences (i.e. bugs) with the way Python works; they should not be apparent in most programs.

Also there's sort of work that will make Python as fast or faster than low-level languages like C/C++. This is called PyPy http://codespeak.net/pypy/dist/pypy/doc/news.html

[Mikko Ohtamaa]

Also there's sort of work that will make Python as fast or faster than low-level languages like C/C++. This is called PyPy http://codespeak.net/pypy/dist/pypy/doc/news.html

If you are worried to speed of Python execution, you have several options
- Use automatic Psycho JIT'ing. Hogs memory quite much. Consider using Psycho only for speed critical modules
- Compile Python to Java bytecode and use Java virtual machine to execute it (JIT). However, binding native functions becomes painful
- IronPython compiles python to .Net Common Language Runtime code
- PyPy isn't here still for a year or two

[Yoghurt]

There won't be speed issue with Python; all computationally expensive parts will be/are written in C++

[Sirus20x6]

Just thoguht I would bring this up.

http://www.gamedev.net/reference/progra ... gmscript1/

A script in GameMonkey is usually a plain-text ASCII file that contains a collection of functions, variable declarations and expressions. Scripts are usually loaded in from disk and compiled into a form that the GM environment can work with, namely bytecode. There is no standard form to a GameMonkey Script; you do not have to import modules like in Python, nor do you have to adhere to any indentation formatting (again, like Python). GameMonkey script does not have a native version of the include directive, so usually a script is self-contained (although you can load and execute multiple scripts on the same machine). The only requirement in GameMonkey Script is that function variables are declared before you try and call them; the ordering of such statements is important because GMScript code is compiled in a single-pass and also it does not have a pre-processor like in C/C++.

[Impaler]

Based on what I have seen over at Civ4 I would strongly support greater use of Python, game logic and the UI are primary candidates. I would be hesitent to have effects data in Python though, I think thats best keept in XML as Civ4 did. As has been noted the AI will need to weight and evaluate thouse raw numbers to come to its conclusions so they should be easily accessible.