So I started creating a Django project on Github with this Travis configuration
language: python python: - "2.6" - "2.7" env: - DJANGO=1.4 install: - pip install -q Django==$DJANGO --use-mirrors - pip install pep8 --use-mirrors - pip install https://github.com/dcramer/pyflakes/tarball/master - pip install coverage - pip install django_coverage before_script: - "pep8 --exclude=migrations --ignore=E501,E225,E261,W191,E101 ." - pyflakes -x W . script: - python manage.py test_coverage
Basicaly I added coverage and django_coverage to the depandecy and then ran the tests with coverage report enabled. ( in real world I would use a requirement file but I will keep dep. installation this way for the sake of clarity) This is the result:
Name Stmts Miss Cover Missing --------------------------------------------------------- trango.molten_core.models 0 0 100% trango.molten_core.views 0 0 100% --------------------------------------------------------- TOTAL 0 0 100%
Ok this is just and empty app with one test always passing. But my point is to put this data somewhere outside Travis ( and gh-pages seemed a very good place ) and have an easy way to see how coverage increase or decrease in particular builds ( and who is guilty for that!)
First we have to find an easy way to identify the build commit. I found that something like
export GIT_COMMIT_ID=`git rev-parse HEAD` works just fine. Then I started speculating what was the optimum ( or at least possible) way to move away the data and I ended up with these three solutions:
HUMAN PALADIN: Make Travis-CI push back the reports
This is the first, clean , way that came to my mind. I created the gh-pages branch on my repo ( if you are unfamiliar with this please read here ) and I set django_coverage to create the html reports in a specific folder (in settings.py file)
if os.getenv('GIT_COMMIT_ID'): COVERAGE_REPORT_HTML_OUTPUT_DIR = os.path.join("reports", os.getenv('GIT_COMMIT_ID')) else: COVERAGE_REPORT_HTML_OUTPUT_DIR = 'reports'
Then I tried to meke Travis-CI to push back the reports
language: python python: - "2.6" - "2.7" env: - DJANGO=1.4 install: - pip install -q Django==$DJANGO --use-mirrors - pip install pep8 --use-mirrors - pip install https://github.com/dcramer/pyflakes/tarball/master - pip install coverage - pip install django_coverage before_script: - "pep8 --exclude=migrations --ignore=E501,E225,E261,W191,E101 ." - pyflakes -x W . - mkdir reports - export GIT_COMMIT_ID=`git rev-parse HEAD` - mkdir reports/$GIT_COMMIT_ID script: - python manage.py test_coverage after_script: - cp -pR reports/$GIT_COMMIT_ID /var/tmp - rm -rf reports/$GIT_COMMIT_ID - git branch gh-pages - cp -pR /var/tmp/$GIT_COMMIT_ID reports - git add reports/* - git commit -m "reports for $GIT_COMMIT_ID" - git push origin gh-pages
I had to move the reports in a temp directory to quietly switch from the master branch to the gh-pages branch. And sadly this is the output of the build
$ git branch gh-pages 103 104$ cp -pR /var/tmp/$GIT_COMMIT_ID reports 105 106$ git add reports/* 107 108$ git commit -m "reports for $GIT_COMMIT_ID" 109[detached HEAD 5b018bc] reports for 4b01202c546546bc424af1828eb211a011bcc278 110 Committer: vagrant <vagrant@nettuno.(none)> 111Your name and email address were configured automatically based 112on your username and hostname. Please check that they are accurate. 113You can suppress this message by setting them explicitly: 114 115 git config --global user.name "Your Name" 116 git config --global user.email email@example.com 117 118After doing this, you may fix the identity used for this commit with: 119 120 git commit --amend --reset-author 121 122 3 files changed, 384 insertions(+), 0 deletions(-) 123 create mode 100644 reports/4b01202c546546bc424af1828eb211a011bcc278/excludes.html 124 create mode 100644 reports/4b01202c546546bc424af1828eb211a011bcc278/index.html 125 create mode 100644 reports/4b01202c546546bc424af1828eb211a011bcc278/modules/trango.molten_core.models.html 126 127$ git push origin gh-pages 128fatal: remote error: 129 You can't push to git://github.com/sammyrulez/django-travis-demo.git 130 Use firstname.lastname@example.org:sammyrulez/django-travis-demo.git 131 132 133 134after_script: 'git push origin gh-pages' returned false. 135Done. Build script exited with: 1
As you can see when Travis-CI tries to commit we got evidence that git is not configured with a proper user nor have keys or Github user to add as collaborator to my repo. So also the push fails. I think up to now this is a dead end. Possible solutions are to run Travis-CI localy ( but I would go straight in Jenkins arms instead: the good point in favor of Travis-CI is his cheap and cloudy nature ) or that Travis-CI allows you to add specific deployments keys to each building repository.
FORSAKEN WARRIOR: Use a curl friendly file hostSearching for options of "push-back" I found a message in the Travis-CI mailing list suggesting to use a file hosting service to send your data via curl. For example ge.tt. Now the problem is that you have to put your API Key, username and clear text password in the .travis.yaml file commited in a public repository. Definitively not a secure way to store files online. One solution would be to create a proxy service that uses commit hashes as ONE TIME PASSWORD. But building such a thing would require a lot of time and the result would not be "vandal proof". Maybe, someday....
ConclusionTravis-CI is an amazing project but without a feature to export build results I find hard to think it would fit a project with many developers and aiming a high code quality and test coverage. I found historical CI data very important and in some case also inspiring ("Look how crappy the project were a month ago!"). Travis-CI is handy and sleek but having reports on the project web page gives an instant direction on how the project is managed. Well also the Travis badge is a really really good indicator.
I will keep it failing just to remember to fix it!