how to execute tests on a bazaar pre-commit hook

you should always execute the test suite from a project before committing the source code to a version control system. everybody knows this but sometimes we forget… so lets have a pre-commit hook that will automatically execute the test suite. i am using bazaar.

in bazaar you write hooks as plugins.

how it works

the “hook” is simple python function that will be executed before really committing when bzr commit is called . it is very simple, it just run an executable and check its result. if it is successful the commit operation is executed. if it fails the commit operation is cancelled.

plugins in bazaar are not project specific. so you cant control in which projects (branches) your plugin will be applied (it will be applied to all). so, first the plugin checks if the branch contains a “precommit” executable. it must be named “precommit” and must be in root path of your branch.

the plugin file should be placed in ~/.bazaar/plugins/. the name of the module doesn’t matter (i called it “check_pre_commit.py”). putting the boilerplate code together i got this:

"""this is a plugin/hook for bazaar. just add this file to ~/.bazaar/plugins/"""

from bzrlib import branch

def pre_commit_hook(local, master, old_revno, old_revid, future_revno, future_revid, tree_delta, future_tree):
    """This hook will execute precommit script from root path of the bazaar
    branch. Commit will be canceled if precommit fails."""

    import os,subprocess
    from bzrlib import errors

    # this hook only makes sense if a precommit file exist.
    if not os.path.exists("precommit"):
        return
    try:
        subprocess.check_call(os.path.abspath("precommit"))
    # if precommit fails (process return not zero) cancel commit.
    except subprocess.CalledProcessError:
        raise errors.BzrError("pre commit check failed.")

branch.Branch.hooks.install_hook('pre_commit', pre_commit_hook)
branch.Branch.hooks.name_hook(pre_commit_hook, 'Check pre_commit hook')

thats all for the hook.

now we just need to create an executable that will run the tests and return something different from zero if it fails. and put it on the root path of the branch.
in this example i execute my tests using nose.

precommit

#!/bin/sh
nosetests

ok. everything is working. but who remember to run the tests should be “punished” and will have to wait for the tests to run twice (once manually before committing and once automatically as a pre-commit hook)? no. if you remember to run the tests you can also remember to skip the plugin :)

bzr commit --no-plugins

Update: You can put your tests in a smart build-system (like doit) to keep track of the modifications in your source code and re-execute only the required tests (the ones that depends on the modified files).

caveats

• the precommit executable is part of the branch but the hook is not. so just doing a branch on the project is not enough. every user has to install the plugin. but just once for all projects.

• the hook is executed after you supply the commit message. it is annoying to write a message and than loose it because the commit failed. or maybe it is a good punishment for trying to commit an untested code :)

• the output gets a bit messed up. but not a big deal…

  eduardo@eduardo-laptop:~/work/doit$ bzr commit
  Committing to: /home/eduardo/work/doit/
  modified precommit
  [==========      ] Running pre_commit hooks [Check pre_commit hook] - Stage 3/5....
  ----------------------------------------------------------------------
  Ran 4 tests in 0.268s
  
  OK
  Committed revision 2.

thats it.

python testing + folder structure – import issues

whenever i start coding something in python i use the following folder structure:

my_project
        ./foo
             ./__init__.py
             ./bar.py
        ./tests
             ./test_foo.py
             ./test_bar.py

but i used to get confused on how to organize the import(s) and when to use PYTHONPATH for my tests.

my requirements are:

• i want to be able to run my tests both from my_project folder and from the tests folder
• i don’t want to set the PYTHONPATH manually (and i don’t want to add my development path to PYTHONPATH.
• i want to import the lib modules on the test on the same way i would import after the lib is deployed.

(yes i am lazy)

bar.py:

def dumb_true():
    return True

1st try

the code for test_bar.py should like something like this:

import unittest

from foo import bar

class TestBar(unittest.TestCase):
    def test_bar_true(self):
        self.assertTrue(bar.dumb_true())

if __name__ == '__main__':
    unittest.main()

the problem here is that you need to set the PYTHONPATH manually.

2nd try

the first thing that comes in my mind is to add ‘foo’ folder to PYTHONPATH on the test module

...
import sys,os
sys.path.insert(0,os.path.abspath(__file__+"/../.."))
from foo import bar
...

not bad. but you need to add this to every single test module. and hope that your folder structure wont change.

3rd try

i guess we need a smarter test runner (like nose).
get your code back to just:

...
from foo import bar
...

you can just run “nosetests” from the project folder (‘my_project’ in the example above) and it will just work.

the same doesn’t happen if run “nosetests” from the tests folder.

from nose homepage:

When nose imports a module, it adds that module’s directory to sys.path; when the module is inside of a package, like package.module, it will be loaded as package.module and the directory of package will be added to sys.path.

so the trick here is to make the tests folder a package (adding __init__.py to it). so in the tests folder you will find:

./tests
     ./__init__.py
     ./test_foo.py
     ./test_bar.py

now from the tests folder you can:

1. execute all tests doing “nosetests”
2. execute just one file “nosetests test_bar.py”
3. check nose docs for more options

and you don’t have to worry about PYTHONPATH at all :)

more nose

nose also free you from all the unittest boilerplate code. the test file could be just:

from foo import bar

def test_bar_true():
    assert bar.dumb_true()

and thats it.

running selenium tests from the command line

I wrote a tutorial on how to run selenium tests from the command line. it is very handy you plan to include it on your continuous integration process.

intro

selenium is great tool for writing tests to web applications. and with Selenium IDE, it is a breeze to write tests. here i will describe how to run your selenium tests from the command line (instead of manually starting the server, opening a browser, selecting the test file…). if you dont use selenium yet you should take a look at it first.

i will not go over the advantages over having regression tests… but to be effective it should be easy to run the tests. it is very boring to manually run the tests, so once in a while when i decide to run it i found out that some regression bugs appeared, and they were not easy to spot because they might got into the code a while ago. even worse is when somebody else on team introduced the bug ;)

complete article