Pushing X.org Git repos to Github et al.

TL;DR;

If you want to know why cloning several of the X.org repositories to Github or GitLab instances fails and how this can be worked around, you may want to continue reading.

Why we stumbled over this issue...

As a joint effort of the Arctica Project, TheQVD and X2Go, Ulrich Sibiller and I are currently preparing a build workflow for the nxagent X-server (version 3) [1] that allows building nxagent against the modular X.org 7.0 (using autoconf and automake) rather than the monolithic build workflow of X.org 6.9 (using ancient imake).

Our goal is to rewind all X.org components required for building nxagent back to a state where nxagent successfully builds and runs. Then we will go through various, (probably) monthly cycles of

bumping commit level of required X.org components
rebuild and test nxagent
fix FTBFS, if any
fix broken functionality, if any

Our first hurdle...

After Ulrich now has a functioning nxagent-against-X.org-7.0-workflow locally, we want to get everything into the Arctica Project's Github namespace. Which fails...

[mike@minobo xorg.upstream]$ git clone https://anongit.freedesktop.org/git/xorg/lib/libX11.git
Cloning into 'libX11'...
remote: Counting objects: 18520, done.
remote: Compressing objects: 100% (3441/3441), done.
remote: Total 18520 (delta 15094), reused 18325 (delta 14954)
Receiving objects: 100% (18520/18520), 5.98 MiB | 549.00 KiB/s, done.
Resolving deltas: 100% (15094/15094), done.
Checking connectivity... done.
[mike@minobo xorg.upstream]$ cd libX11/
[mike@minobo libX11 (master)]$ git remote rename origin upstream 
[mike@minobo libX11 (master)]$ git remote add origin git@github.com:ArcticaProject/libX11.git
[mike@minobo libX11 (master)]$ git push --mirror origin 
Counting objects: 18520, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3301/3301), done.
remote: error: object 70d5e4d45dd7bf1e05b099cb5a4dd529344084f0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
remote: fatal: Error in object
error: pack-objects died of signal 13
error: failed to push some refs to 'git@github.com:ArcticaProject/libX11.git'

This issue should be a known issue at X.org / Mesa upstream [2].

The underlying problem occurs with Git 2.5 and above...

If you run git-fsck from Git (>= 2.5) against several of the X.org Git repositories, you will stumble over the above issue. For libX11.git, we see these errors reported when running git-fsck from current Debian unstable:

mike@sid:~/xorg/libX11.git$ git fsck 
Checking object directories: 100% (256/256), done.
error in tag 70d5e4d45dd7bf1e05b099cb5a4dd529344084f0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a3bfe698090a5d41f1e9acb1b57a049085d6b04e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag c1fc8c9aec1bc92d03d08d5e986dd40b194a7a3e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag d4c27f7e7d8e2ea32418f341ad85c33f3b76862d: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 79780379aee21bf4d4bcb046a3b54774893ea6b1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a638de80ec20dc1de625cd323ed19a6646fecf2e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag f6a6414e4e52a971a35fd118c350567e2383d034: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b5c6810e21be2e7ccfac8f0539d46bf75dbe50a0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 86cec9d20428cc190ec7278a0abe481b180288ac: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag ac912988091574790c7cffbbc2c60b8c59f8fcef: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2d01674092951cd3284b3b099978b2436ea468ad: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 922d0534154668918eaa09a36cc5cd53fc6b71b7: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 8ef91b9cc0a4a7b0361f6b40b3f735221c8272cb: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 1516173727005a87aa501e1ad708d72e6ae6e753: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 77c9e63a3abff1593ccbed249b2c3ae7f68e1833: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 58044c7087137ce2b521297dbb31934ccaedc94e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2e0d061c6830891dcc856a04aac900e5bb6d0779: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2f58c7cc4cbf3d30927f6301039b6bf018ed38fd: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 77fc2e1e26019c3ba92275d249fe1979570255d2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9b2e06fa262f23dbcbbccd42be6477e08a452268: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 7f99410bb9cc9f7cbbc2d43d8ad044771a6eec0a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 99b59b83f126b4abb9da89f577a5b8147d543c25: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 97170c33f99761acd932f968e21d6d7664e7ba80: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b39966d452127c992987d082f69321f950ae4c39: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 05e20c3bd8ac6b7e873607db73894486a4ac0519: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag f478497b5a3e9d6b92237e0f408c9f0f18108f52: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag c5ed936f28ce70c8932e33d67a6e06d140e35ba8: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2b6beba295a6ff997dfe15a732c95ae6577fa735: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 32edbb1e9c5b7d1c6f8639dab85e5368d303df7f: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 490fbe2afb9ffc7057c0eb150f7cdd4d4fa8686c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag fc605ac48bd7d675d6d04162a0f3bfebe27f2037: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 6289ecc26d1d505f395c59da744a9354d7aad2a2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 8c9c30dac72aebd1a2c31a55a47c0ac11bc328db: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 3b7aa57994924be0dad693e837559d9ee900299c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag b02ad9a8ec96e3c1027ca71289aeef5b8748e17e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 4402ab1a86226a9cd0ce84fb8147b1c4958c684d: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag bb02b4fbde316644c771d28d5edeeb9b213a6d6a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 95b6462c70213b562c9d2b8726eda706edc9c456: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag a99fea43ad53907623f269942c55237c238645e0: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 0c417ce98a6cf927ce7a1359b198b2bb746c707e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag ce3ff6b102f360fc9dcd3a85361f44da054de0b1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2fb7b16b38a365c5009bd170d4463634c1f3de26: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 95570f484bfde5fc2ed8548cb17b87d8114a2866: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag d6e5895c104cfe0d135494a605013dd2910a93a1: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 49be7144523c4a56afcb389a9aa021d167710d2c: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 020353d1b982ac938d229039ca13bfea5793fcd9: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 991792d4f8477ab65ad4d8a0245c9a3bbb1f0294: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag e8037555efe299a7143370b485f754f6b08ffac7: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 4fdc8e36a5022f3e32d5567305d8e9cde4011f1a: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9c93a19a76163328e30a7306bb0babe3175fb9a2: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 9f90bf4424250f80340b3e2803d343c74d2adb0b: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 2e7a8c7974777e3e869de9013025dc3f61633e1e: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 59b7c06da2a64d72668394e9239e3dfe8a5cff59: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 0263c4f684faa9a5126a0db524d5d35be7277e52: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 1514e654c1a9b346cbfca5a81f6d331f106242aa: missingSpaceBeforeDate: invalid author/committer line - missing space before date
error in tag 6aa83ec3d1d82ee332cedcf4e99d3cbd56e73edd: missingSpaceBeforeDate: invalid author/committer line - missing space before date
Checking objects: 100% (18523/18523), done.

What actually is going wrong here...

When looking at one of those tags (luckily, only tags are affected), we see that something's not ok with the date string of the tag (I am not a Git plumber, so sorry for being superficial here):

mike@sid:~/xorg/libX11.git$ git show 2fb7b16b38a365c5009bd170d4463634c1f3de26
tag XORG-RELEASE-1-TM-MERGE
Tagger: Alan Coopersmith 
Date:   Thu Jan 1 00:00:00 1970 +0000
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

commit 84adc60cae8e944777563501dd633622a28bdb3b
Author: Alan Coopersmith 
Date:   Sat Mar 27 07:08:16 2004 +0000

    Fix typo (IsModiferKey should be IsModifierKey)

[... not quoting the diff for the above commit ...]

How to work around that issue...

We then got the idea that we could redo those tags with the date string derived from the commit that this tag points at.

This can be scripted (credits go to an unnamed person in Ulrich's surrounding!!!):

mike@sid:~/xorg/libX11$ cat repair.sh
#!/bin/bash

set -x

git fsck 2>&1 | tee git-fsck.log | grep "error in tag" | sed -r -e 's/error in tag ([0-9a-f]+):.*/\1/p' | while read broken; do
         echo $broken
         commit=$(git rev-parse ${broken}^{commit})
         tag=$(git cat-file tag $broken | sed -n 's/^tag //p')
         tag_msg=$(git cat-file tag $broken | sed -n '/^$/,$p' | tail -n +2)
         export GIT_COMMITTER_NAME="$(git log -1 --format='%cn' $commit)"
         export GIT_COMMITTER_EMAIL="$(git log -1 --format='%ce' $commit)"
         export GIT_COMMITTER_DATE="$(git log -1 --format='%cD' $commit)"
         git tag -a -f -m "$tag_msg" $tag $commit
done
# drop all old content...
git reflog expire --expire=all --all
git gc --prune=all
# no more errors should be reported...
git fsck

Copy this repair.sh into the Git repo itself and let the script walk over your tags. After this script has been applied you can push the X.org Git repo with broken tags redone to Git hosters like Github or GitLab.

light+love,
Mike

[1] https://github.com/ArcticaProject/nx-libs
[2] https://lists.freedesktop.org/archives/mesa-dev/2016-February/106268.html

By sunweaver at 03/30/2016 - 15:24
sunweaver's blog
Login to post comments

sunweaver's NET