Page 1 of 2

Build number issue

Posted: Wed Jul 15, 2015 7:07 pm
by Vezzra
In commit 3323243 this:

Code: Select all

    commit = check_output(["git", "show", "-s", "--format=%h", "HEAD"]).strip()
    timestamp = float(check_output(["git", "show", "-s", "--format=%ct", "HEAD"]).strip())
    build_no = ".".join([datetime.utcfromtimestamp(timestamp).strftime("%Y-%m-%d"), commit])
...has been simplified to:

Code: Select all

    build_no = check_output(['git', 'show', '-s', '--format=%cd.%h', '--date=short', 'HEAD'], shell=False).strip()
However, with that change the conversion of the timestamp to UTC has been removed.

My first concern was that the "git show" command might convert the committer date to local time, which of course would have resulted in possible different build numbers, depending on the time zone of the computer on which a build is produced. However, this isn't the case, as the "git show" command returns the timestamp in the local time of the original committer. Which in turn creates another problem: If, for example, Dilvish (who is at -0700) makes a commit 2 hours after me (I'm at +0200, so 9 hours ahead), his commit, although the later one, has an "earlier" timestamp. Resulting in a possibly "lower" build number for the later commit.

That of course defeats the whole point of composing our build number the way it's done now, which is to have consecutive, incremental build numbers. Therefore, as a quick solution, I suggest to revert back to the original implementation which converts the timestamp to UTC before using it for the build number. This should take care of the issue.

I've created a PR with the respective changes.

But this issue illustrates quite nicely that using a date as build number isn't the most brilliant idea I ever had (Marcel and I had a brief exchange about that via PM). So I'm once again looking for a better way to compose our build number. Suggestions welcome, the best I've come up so far is the following:

Tag the very first commit of the repo with an annotated tag like "build-no-base" and simply use "git describe --match build-no-base" to get the total number of commits. Use this as the first part of the build number, and the hash as the second part (like it's already done now).

Comments?

Re: Build number issue

Posted: Fri Jul 24, 2015 3:07 pm
by Vezzra
Continuing a discussion started here, but fits better here in this thread:
Chriss wrote:The "what version number to use" question is probably not specific to arch...
Yep, we've already had issues with our current solution for composing the build number, see above. The "git rev-list --count HEAD" command in your OP in this thread although would accomplish the same as my suggestion to tag the first commit with "build-no-base" and use "git describe --match build-no-base" to get the total number of commits, but without the need for the tag.

That looks far better (and not to mention, more compatible with FO packages for Linux distros) than what we have now, but so far I've got no comments on my suggestion. The change can be made quite quickly (still in time for 0.4.5), but this is not my decision alone...

Geoff, Dilvish, Mat, Marcel?

Re: Build number issue

Posted: Fri Jul 24, 2015 3:20 pm
by adrian_broher
Vezzra wrote: […] but so far I've got no comments on my suggestion
[…] Marcel?
Still don't care and still see the git hash only as sufficient.

Re: Build number issue

Posted: Fri Jul 24, 2015 3:57 pm
by MatGB
It's one of those "above my paygrade" things, I like having the date in it, but always managed perfectly with the SVN number, the hash isn't as clear as a rising number but if that's easiest it's fine, at least for the test builds, probably best to have some sort of date in the release builds but not totally necessary.

Re: Build number issue

Posted: Fri Jul 24, 2015 6:03 pm
by Dilvish
I seem to have a slight preference for the date over the simple commit count, but it seems a pure personal preference (and a minor one at that) with on particular supporting reason I can identify.

Re: Build number issue

Posted: Fri Jul 24, 2015 6:19 pm
by Dilvish
Thinking a bit more on the topic, I should add-- my thinking and comment thus far had really been more around the build number to use in our regular weekly builds, not for the 'Stable Release'. For the latter I could certainly understand wanting to drop the date.

On the total commit count thing, though, I am concerned it is not really monotonically increasing like I have the impression you consider it to be. I believe that Git garbage collection can cause the number to suddenly and unexpectedly drop, and of course it would also vary tremendously by person depending on how many PRs you had downloaded or other experimental branches you had.
Eventually, Git will decide that it's time to run garbage collection. (You can trigger this process yourself, using git gc.) Starting from every branch and every tag, Git walks back through the graph, building a list of every commit it can reach. Once it's reached the end of every path, it deletes all the commits it didn't visit.

Re: Build number issue

Posted: Fri Jul 24, 2015 6:44 pm
by adrian_broher
Dilvish wrote:I believe that Git garbage collection can cause the number to suddenly and unexpectedly drop, and of course it would also vary tremendously by person depending on how many PRs you had downloaded or other experimental branches you had.
I don't follow you how you came to this conclusion that git rev-list does not return a monotonic increasing number.

quoting the manual:
git-rev-list wrote:List commits that are reachable by following the parent links from the given commit(s)
and
git-gc wrote: Runs a number of housekeeping tasks within the current repository, such as compressing file revisions (to reduce disk space and increase performance) and removing unreachable objects which may have been created from prior invocations of git add.
TL;DR:
git-rev-list only counts reachable commits.
The Git GC deletes only unreachable commits.

Re: Build number issue

Posted: Fri Jul 24, 2015 8:32 pm
by Dilvish
adrian_broher wrote:git-rev-list only counts reachable commits.
The Git GC deletes only unreachable commits.
From my reading, unless you add --all (or perhaps --branches?) to your rev-list command, then these lines would not actually talking about the same partition of reachable/unreachable". If you do add such an option, then this summary would support my point, since adding and deleting branches and whatnot is exactly the kind of user action I referred to as changing the total commit count.

Furthermore, if an explicit reference to "git-rev-list" was in this thread before this, I hadn't noticed it. I was responding to
and simply use "git describe --match build-no-base" to get the total number of commits
Even looking into git describe a little I didn't immediately see how that was going to give a number of commits, so I just focused on "the total number of commits"

Re: Build number issue

Posted: Fri Jul 24, 2015 8:44 pm
by adrian_broher
Dilvish wrote:From my reading, unless you add --all (or perhaps --branches?) to your rev-list command, then these lines would not actually talking about the same partition of reachable/unreachable". If you do add such an option, then this summary would support my point, since adding and deleting branches and whatnot is exactly the kind of user action I referred to as changing the total commit count.
Of course you don't want to count commits on branches and only want to count parent/ancestor commits relative to the commit you declare as starting point (HEAD in this case) to the initial commit.

Re: Build number issue

Posted: Fri Jul 24, 2015 9:56 pm
by Chriss
adrian_broher wrote:
Vezzra wrote: […] but so far I've got no comments on my suggestion
[…] Marcel?
Still don't care and still see the git hash only as sufficient.
To describe the code someone refers to, the hash is perfect. BUT, for packaging, I need to have a version which can be ordered. I (or rather the package manager) need to know which package is newer, so I know when to update the package. The Git hash has no order. It is useless for that use case. For packaging, it is NOT sufficient.
Dilvish wrote:
adrian_broher wrote:git-rev-list only counts reachable commits.
The Git GC deletes only unreachable commits.
From my reading, unless you add --all (or perhaps --branches?) to your rev-list command, then these lines would not actually talking about the same partition of reachable/unreachable". If you do add such an option, then this summary would support my point, since adding and deleting branches and whatnot is exactly the kind of user action I referred to as changing the total commit count.
As far as I understand this: By specifying a commit, you specify a branch and everything else. You refer to one specific state of the code. Yes, your total commit count in your local repository may vary, but then you're on a different branch, and / or have more commits, so you don't refer to the same state of the code.

You don't even really want to count all commits in your repo. All you really need to count is the number of commits it took to reach that specific state of the code. That count can even restart if it is prefixed by a version tag like one of my examples does, and that version tag increments in a way that the package manager understands. So maybe a Build number could be "0.4.4+r<commit-count>.<git hash>"? Not sure if the "+" works, as in: the relevant package managers understand that 0.4.4 < 0.4.4+ < 0.4.5.

Re: Build number issue

Posted: Fri Jul 24, 2015 10:26 pm
by Dilvish
Chriss wrote:So maybe a Build number could be "0.4.4+r<commit-count>.<git hash>"? Not sure if the "+" works, as in: the relevant package managers understand that 0.4.4 < 0.4.4+ < 0.4.5.
How about simply "0.4.4.<commit-count>.<git hash>"? (and dropping the ".<git hash>" for the stable releases.) If we have the commit count in there I see no need anymore for the "+", nor the "r" really either.

Re: Build number issue

Posted: Sat Jul 25, 2015 7:11 am
by Geoff the Medio
Dilvish wrote:...I see no need anymore for the "+", nor the "r" really either.
+ is still useful, to indicate that it's after the previously-indicated release, without requiring people to know what commit id or count corresponds to what time point / release or compare the numbers.

Re: Build number issue

Posted: Sat Jul 25, 2015 2:16 pm
by Chriss
AFAIK, the r is something which arch uses so the revision can be recognized better. I can add that if you want to drop it.

Re: Build number issue

Posted: Sun Jul 26, 2015 3:44 pm
by Vezzra
Dilvish wrote:I just focused on "the total number of commits"
Sorry, unclear wording on my part. What I meant was that:
adrian_broher wrote:Of course you don't want to count commits on branches and only want to count parent/ancestor commits relative to the commit you declare as starting point (HEAD in this case) to the initial commit.
Which is, as far as I understand, exactly what "git rev-list --count <commit-ref>" does. So the output of that command should yield a stable result independent on what branches that do not lead to the referenced commit the local repo contains. Meaning, it should produce the same result regardless of who builds a certain commit, as the parent/ancestor commit history should be the same.

Which should serve our purpose just fine.

Re: Build number issue

Posted: Sun Jul 26, 2015 4:05 pm
by adrian_broher
Vezzra wrote:
adrian_broher wrote:Of course you don't want to count commits on branches and only want to count parent/ancestor commits relative to the commit you declare as starting point (HEAD in this case) to the initial commit.
Which is, as far as I understand, exactly what "git rev-list --count <commit-ref>" does.
You understood that correctly.