But let start with a short overview of what is Continuous Integration, how it can help us making a better game and how the current infrastructure looks like:
What is Continuous Integration?
Continuous Integration (CI) is a term for the permanent and complete rebuild of a project from source in a clean build environment which is started after every source code change. In its most basic form this can be done by the developer himself before committing his code changes any may be limited to only rebuilding the application. However this is a very tedious task for the developer: Cleaning the build environment and rebuilding the whole application may be be complex and take some time, which can be better utilized for actual designing/development/bug hunting. Because of that the task of CI is usually done by a build server. The build server is some computer that does nothing else than listening to source code changes in a repository, rebuilding the application in a clean build environment and reporting the build status back to the developers.
CI isn't just limited to rebuilding the project and reporting compile errors. As long as you can write some test or check that can work without human intervention it can be used in CI. This can include, but is not limited to:
- Unit testing (Does a small and isolated piece of code conform with some assertions written by developer)
- Integration testing (Does a bigger and interacting piece of code conform with some assertions written by developer ;) )
- Code Linting (Does the code follow a formatting style?)
- Static code analysis (Is there some dubious code use, which may is a certain indicator for a hidden bug? E.g for C++: iterator use after erase, double free, null pointer dereferencing)
- Code test coverage (Do the tests actually cover relevant pieces of application code, are there untested branches? If so, how much is not covered?)
- Documentation coverage (Does documentation exist for a function, are all parameter documented? Are special code cases (e.g. Exception thrown) documented? Are the cross references valid?)
- File asset format validation (Is the added texture a PNG file with 8 bit channels and either greyscale, RGB or RGBA?)
On top of that CI may help with the software delivery (release). As a CI build is done from a clean build environment there it is way harder to create broken releases. Also a CI build may trigger not only on code changes, but on time events or code tagging too. This does allow regular releases on a scheduled time (nightly builds/weekly builds) or on tagging (release tags) without much human intervention.
What CI tools does FreeOrion use?
Currently we rely on several free services and tools to implement the CI. Those are:
- GitHub - The place were we host the code on and which triggers via a webhook (HTTP requests send by GitHub on certain events) other services whenever a commit is done or a pull request is submitted.
- TravisCI - A CI hoster, which provides virtual machines for the event of building source code. Travis provides Ubuntu and MacOSX based virtual machines to build the code and a web interface to both review the build reports and stop/rerun a build from the same commit. The configuration is done via a file called `.travis.yml` file in the root of the source repository, which is a conglomerate of configuration values and bash shell code snipplets.
- AppVeyor - Another CI hoster, which does essentially the same as Travis, but provides virtual machines for Windows. The configuration is done via a file called `appveyor.yml` file in the root of the source repository, which is a conglomerate of configuration values and powershell code snipplets.
- Docker - This is a virtualization software for applications, which does allows users to describe Linux virtual machine image by writing installation instructions for said image. It also consists of a hosting service to store those precomposed VM images and make them available for download. We use this tool to compose a virtual machine for Linux, which contains all build dependencies for FreeOrion. TravisCI by itself provides a rather slim and conservative designed virtual machine image of Ubuntu, which doesn't allow a somewhat recent C++11 build environment + all the special tools we would need for more complex CI tasks.
- freeorion-bot - This is a GitHub/Docker account, solely created for the purpose of interacting with the services mentioned above. Currently when a build happens it is done on behalf of the freeorion-bot, not a certain developer. This will become more useful when weekly builds are a thing or then the CI will do semi-automatic code reviews.
Currently we have the bare essentials in place.
For the `freeorion/freeorion` repository we do:
- Build every commit/commit series and pull requests on Windows, Ubuntu 16.04 and MacOSX 10.10.
- Report build issues on commits/commit series and pull requests on the GitHub web page and via developer eMail.
- Rebuild C++ code documentation for every commit on `master` branch and upload it to http://freeorion.github.io/cppapi/master/
- Build checks itself are limited to whether a build successfully compiles or not.
For the `freeorion/freeorion-sdk` repository we do:
- Build every commit/commit series and pull requests on Windows and MacOSX 10.10, create a Docker image for `freeorion/freeorion` and update it if necessary.
- Report build issues on commits/commit series and pull requests on the GitHub web page and via developer eMail.
- Build checks itself are limited to whether a build successfully compiles or not.
- Create release draft of the FreeOrionSDK from the build artifacts whenever a developer tags a release (in case you ever wondered why `freeorion-bot` is the one releasing the SDK and not me).
As open source project we don't have any other investment than our interest, the knowledge and our time. This means we're pretty much limited to whatever we can get free of charge. All of GitHub, TravisCI and AppVeyor provides their services free of charge for open source projects but limit the use of said service. For example TravisCI limits the time to build to 50 minutes. If a build exceeds this time limit it is killed off. AppVeyor limits this to 60 minutes. When adding CI tools we need to keep this in mind. Also I would consider it a sign of good will to reduce the build time whenever possible, may it be by optimizing the code, the build or by other means and also keep in mind that we share these resources with other open source projects.
Another point are the capabilities of the CI in general. As already mentioned CI is limited to checks, which work non-interactive. When considering a new great tool for CI keep always in mind that the tool MUST work without a user typing commands into a shell or a user clicking away error messages because the virtual machines don't expose any user interface.
Also (but this is a personal interprecation of me) CI should never modify the project. That means the CI service should NEVER commit to the source or modify history. A CI should report issues, but not try to fix them. Maybe the CI does a dumb decision when trying to fix a bug, maybe the CI is buggy itself. The CI is a helper for tiresome tasks and a constant reminder for things that may cause problems, but the final decision is up to the developer.
If there are more questions regarding CI feel free to ask.