Python3 migration

Programmers discuss here anything related to FreeOrion programming. Primarily for the developers to discuss.

Moderators: Committer, Committer

Message
Author
User avatar
Cjkjvfnby
AI Contributor
Posts: 480
Joined: Tue Jun 24, 2014 9:55 pm

Re: Python3 migration

#31 Post by Cjkjvfnby » Wed Jan 15, 2020 8:48 pm

My progress so far.

PR with tests compatiable with Python 3 is ready for review. https://github.com/freeorion/freeorion/pull/2711

TODO:
  • Work with problem places reported by pylint (Here is an example. https://github.com/freeorion/freeorion/pull/2709). Blocked by pr with tests.
  • Setup mypy and check problem places. (It has different check than pylint) Blocked by pr with tests.
  • Add flake8 for python 3 code checks, we will need it after migration. Probably blocked by pr with tests.
PS. I will be away this weekend.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#32 Post by Vezzra » Fri Jan 17, 2020 11:29 am

adrian_broher wrote:
Tue Jan 14, 2020 8:27 am
FYI the majority of Linux Platforms are dropping Python2 with their next release. MacOS will drop support with 10.16.
Which basically means we're actually way overdue with the migration to Python3, and 0.4.9 won't be included in the repos for upcoming Linux distro releases.

Well, not much we can do about that, but the more urgent it will be to get the migration done within the next release cycle.

I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#33 Post by Vezzra » Fri Jan 17, 2020 11:31 am

o01eg wrote:
Tue Jan 14, 2020 9:13 am
Considering https://fedorapeople.org/groups/schedul ... tasks.html looks like freeorion will be missed in Fedora 32.
Here we go...

o01eg
Programmer
Posts: 860
Joined: Sat Dec 10, 2011 5:46 am

Re: Python3 migration

#34 Post by o01eg » Fri Jan 17, 2020 11:46 am

Vezzra wrote:
Fri Jan 17, 2020 11:29 am
I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?
I would like to get Goverment merged and make empire readiness refactoring.
Gentoo Linux x64, gcc-9.2, boost-1.72.0
Ubuntu Server 18.04 x64, gcc-7.4, boost-1.65.1
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2020-07-01.b4f9ea1.
Donations are welcome: BTC:14XLekD9ifwqLtZX4iteepvbLQNYVG87zK

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#35 Post by Vezzra » Fri Jan 17, 2020 11:58 am

o01eg wrote:
Fri Jan 17, 2020 11:46 am
I would like to get Goverment merged and make empire readiness refactoring.
The Government branch is about to be merged, as far as I understand, so that I expect to happen anyway.

Regarding the empire readiness refactoring, as far as I understood, that is supposed to address certain issues, so can be seen as a bug fixing effort. Those can and should go in as well of course.

But aside from such things it might be a good idea to get another release out with the Python3 migration dealt with as soon as possible (by summer maybe?).

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12675
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Python3 migration

#36 Post by Geoff the Medio » Fri Jan 17, 2020 1:16 pm

Vezzra wrote:
Fri Jan 17, 2020 11:58 am
o01eg wrote:
Fri Jan 17, 2020 11:46 am
I would like to get Goverment merged and make empire readiness refactoring.
The Government branch is about to be merged, as far as I understand, so that I expect to happen anyway.
I'd like to get the Government branch merged soon as well, and could just click twice to do that. But realistically it's a substantial new feature that will take a lot of time to get adequately fleshed out with content, balanced, and integrated with the AI.

Given the importance of getting an update with Python3 support, and given that substantial progress towards that is currently happening, I'm somewhat inclined to hold off on the Government merge and to try to get a Python3 update out as soon as possible after v0.4.9.

It's a bit of a hassle to keep the branch separate, but interested people can check out the Government branch now and start providing feedback and help with further development of it without merging into master.

Ophiuchus
Programmer
Posts: 1520
Joined: Tue Sep 30, 2014 10:01 am
Location: Wall IV

Re: Python3 migration

#37 Post by Ophiuchus » Fri Jan 17, 2020 1:28 pm

Geoff the Medio wrote:
Fri Jan 17, 2020 1:16 pm
It's a bit of a hassle to keep the branch separate, but interested people can check out the Government branch now and start providing feedback and help with further development of it without merging into master.
I could ship fresh government based snaps if v0.4.9 is out in parallel to the master weekly builds.
But probably only if somebody explicitly asks for it - i never got any feedback on the builds i did in the past.
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.

Furthermore, I propse... we should default to four combat rounds instead of three ...for the good of playerkind.

o01eg
Programmer
Posts: 860
Joined: Sat Dec 10, 2011 5:46 am

Re: Python3 migration

#38 Post by o01eg » Fri Jan 17, 2020 1:35 pm

Vezzra wrote:
Fri Jan 17, 2020 11:29 am
adrian_broher wrote:
Tue Jan 14, 2020 8:27 am
FYI the majority of Linux Platforms are dropping Python2 with their next release. MacOS will drop support with 10.16.
Which basically means we're actually way overdue with the migration to Python3, and 0.4.9 won't be included in the repos for upcoming Linux distro releases.

Well, not much we can do about that, but the more urgent it will be to get the migration done within the next release cycle.

I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?
Technically I suppose it possible to backport Python-only changes to release branch and make 0.4.9.1 release with python2->3 port. They will be network compatible, but default/ files from 0.4.9 won't be compatible with default/ files from 0.4.9.1.
Gentoo Linux x64, gcc-9.2, boost-1.72.0
Ubuntu Server 18.04 x64, gcc-7.4, boost-1.65.1
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2020-07-01.b4f9ea1.
Donations are welcome: BTC:14XLekD9ifwqLtZX4iteepvbLQNYVG87zK

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#39 Post by Vezzra » Sun Jan 19, 2020 2:34 pm

Geoff the Medio wrote:
Fri Jan 17, 2020 1:16 pm
I'd like to get the Government branch merged soon as well, and could just click twice to do that. But realistically it's a substantial new feature that will take a lot of time to get adequately fleshed out with content, balanced, and integrated with the AI.

Given the importance of getting an update with Python3 support, and given that substantial progress towards that is currently happening, I'm somewhat inclined to hold off on the Government merge and to try to get a Python3 update out as soon as possible after v0.4.9.
That sounds reasonable, and if you prefer to hold off on (bigger) new features and do a quick release cycle focused on the migration to Phyton3, I'm very much inclined to second that.

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#40 Post by Vezzra » Sun Jan 19, 2020 2:48 pm

o01eg wrote:
Fri Jan 17, 2020 1:35 pm
Technically I suppose it possible to backport Python-only changes to release branch and make 0.4.9.1 release with python2->3 port. They will be network compatible, but default/ files from 0.4.9 won't be compatible with default/ files from 0.4.9.1.
In theory that is of course possible, but I don't think it's really feasible. On Windows and macOS we do not rely on a Python environment provided by the OS (as it's the case on Linux), but ship the Python framework with the installer. Also, the SDKs for Windows and macOS have the Python framework packaged.

That means, to do a 0.4.9.1 release with Python3 we need to switch SDKs on the release branch. This is something I definitely do not want to do. Changes that are big enough to require a new SDK version go beyond the scope of bugfix releases and should require a new major release.

Aside from that, the switch to Python3 requires changes to practically all Python scripts (AI, universe generation and turn events). Separating these adjustments from all the other changes to the Python scripts that went into master (and those that most likely will go into master) after the 0.4.9 release branch had been created, to cherry pick them to the release branch sounds like a nightmare. One I do not want to deal with.

I have considered that option myself (backporting the Python3 migration to the 0.4.9 release branch), but for the reasons cited I discarded that idea.

User avatar
Cjkjvfnby
AI Contributor
Posts: 480
Joined: Tue Jun 24, 2014 9:55 pm

Re: Python3 migration

#41 Post by Cjkjvfnby » Thu Jan 23, 2020 7:23 pm

I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#42 Post by Vezzra » Fri Jan 24, 2020 1:50 pm

Cjkjvfnby wrote:
Thu Jan 23, 2020 7:23 pm
I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?
I see. Well, I can try to squeeze it in... depending on if we declare the official release on Sunday. If we do, you'll get new test builds of master on Tuesday anyway. If not, I can see what I can do.

You're on Windows, right?

User avatar
Cjkjvfnby
AI Contributor
Posts: 480
Joined: Tue Jun 24, 2014 9:55 pm

Re: Python3 migration

#43 Post by Cjkjvfnby » Tue Jan 28, 2020 8:25 pm

Vezzra wrote:
Fri Jan 24, 2020 1:50 pm
Cjkjvfnby wrote:
Thu Jan 23, 2020 7:23 pm
I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?
I see. Well, I can try to squeeze it in... depending on if we declare the official release on Sunday. If we do, you'll get new test builds of master on Tuesday anyway. If not, I can see what I can do.

You're on Windows, right?
Yes I am on windows.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Vezzra
Release Manager, Design
Posts: 5290
Joined: Wed Nov 16, 2011 12:56 pm
Location: Sol III

Re: Python3 migration

#44 Post by Vezzra » Sun Feb 02, 2020 1:39 pm

Sorry for the delay, have been quite busy last week.

I provided an inofficial test build (build of latest master as of the time of this post), I uploaded the binaries only package (assuming you don't need the full installer):

https://sourceforge.net/projects/freeor ... p/download

User avatar
Cjkjvfnby
AI Contributor
Posts: 480
Joined: Tue Jun 24, 2014 9:55 pm

Re: Python3 migration

#45 Post by Cjkjvfnby » Sun Feb 09, 2020 6:01 pm

Migration to python3 will be delivered in the next release.

Do we plan to release any other features?

I try to list all the required actions to get it ready:
  • Python 2, 3 compatibilities.
    I am working on it. I think we can do it in a week.
  • C++ changes.
    Looks like it is almost done for AI, what about UniverseGeneration and TurnEvents?
  • Linux, mac builds.
    Do we need to do something here or it will work from the box?
  • Windows build.
    We need to ship python binaries together with the installer.
  • Remove compatibility from the python code
    I don't know if we want to do it in the release or later.
  • Linux distributives
    Some distributives excluded FreeOrion from their repositories, need to fix that. I am not familiar with that world.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

Post Reply