how to execute tests on a bazaar pre-commit hook

20 01 2008
you should always execute the test suite from a project before committing the source code to a version control system. everybody knows this but sometimes we forget… so lets have a pre-commit hook that will automatically execute the test suite. i am using bazaar.

in bazaar you write hooks as plugins.

how it works

the “hook” is simple python function that will be executed before really committing when bzr commit is called . it is very simple, it just run an executable and check its result. if it is successful the commit operation is executed. if it fails the commit operation is cancelled.

plugins in bazaar are not project specific. so you cant control in which projects (branches) your plugin will be applied (it will be applied to all). so, first the plugin checks if the branch contains a “precommit” executable. it must be named “precommit” and must be in root path of your branch.

the plugin file should be placed in ~/.bazaar/plugins/. the name of the module doesn’t matter (i called it “”). putting the boilerplate code together i got this:

"""this is a plugin/hook for bazaar. just add this file to ~/.bazaar/plugins/"""

from bzrlib import branch

def pre_commit_hook(local, master, old_revno, old_revid, future_revno, future_revid, tree_delta, future_tree):
    """This hook will execute precommit script from root path of the bazaar
    branch. Commit will be canceled if precommit fails."""

    import os,subprocess
    from bzrlib import errors

    # this hook only makes sense if a precommit file exist.
    if not os.path.exists("precommit"):
    # if precommit fails (process return not zero) cancel commit.
    except subprocess.CalledProcessError:
        raise errors.BzrError("pre commit check failed.")

branch.Branch.hooks.install_hook('pre_commit', pre_commit_hook)
branch.Branch.hooks.name_hook(pre_commit_hook, 'Check pre_commit hook')

thats all for the hook.

now we just need to create an executable that will run the tests and return something different from zero if it fails. and put it on the root path of the branch.
in this example i execute my tests using nose.



ok. everything is working. but who remember to run the tests should be “punished” and will have to wait for the tests to run twice (once manually before committing and once automatically as a pre-commit hook)? no. if you remember to run the tests you can also remember to skip the plugin :)

bzr commit --no-plugins

Update: You can put your tests in a smart build-system (like doit) to keep track of the modifications in your source code and re-execute only the required tests (the ones that depends on the modified files).


  • the precommit executable is part of the branch but the hook is not. so just doing a branch on the project is not enough. every user has to install the plugin. but just once for all projects.

  • the hook is executed after you supply the commit message. it is annoying to write a message and than loose it because the commit failed. or maybe it is a good punishment for trying to commit an untested code :)

  • the output gets a bit messed up. but not a big deal…

    eduardo@eduardo-laptop:~/work/doit$ bzr commit
    Committing to: /home/eduardo/work/doit/
    modified precommit
    [==========      ] Running pre_commit hooks [Check pre_commit hook] - Stage 3/5....
    Ran 4 tests in 0.268s
    Committed revision 2.

thats it.

python testing + folder structure – import issues

19 01 2008
whenever i start coding something in python i use the following folder structure:


but i used to get confused on how to organize the import(s) and when to use PYTHONPATH for my tests.

my requirements are:

  • i want to be able to run my tests both from my_project folder and from the tests folder
  • i don’t want to set the PYTHONPATH manually (and i don’t want to add my development path to PYTHONPATH.
  • i want to import the lib modules on the test on the same way i would import after the lib is deployed.

(yes i am lazy)

def dumb_true():
    return True

1st try

the code for should like something like this:

import unittest

from foo import bar

class TestBar(unittest.TestCase):
    def test_bar_true(self):

if __name__ == '__main__':

the problem here is that you need to set the PYTHONPATH manually.

2nd try

the first thing that comes in my mind is to add ‘foo’ folder to PYTHONPATH on the test module

import sys,os
from foo import bar

not bad. but you need to add this to every single test module. and hope that your folder structure wont change.

3rd try

i guess we need a smarter test runner (like nose).
get your code back to just:

from foo import bar

you can just run “nosetests” from the project folder (‘my_project’ in the example above) and it will just work.

the same doesn’t happen if run “nosetests” from the tests folder.

from nose homepage:
When nose imports a module, it adds that module’s directory to sys.path; when the module is inside of a package, like package.module, it will be loaded as package.module and the directory of package will be added to sys.path.

so the trick here is to make the tests folder a package (adding to it). so in the tests folder you will find:


now from the tests folder you can:

  1. execute all tests doing “nosetests”
  2. execute just one file “nosetests”
  3. check nose docs for more options

and you don’t have to worry about PYTHONPATH at all :)

more nose

nose also free you from all the unittest boilerplate code. the test file could be just:

from foo import bar

def test_bar_true():
    assert bar.dumb_true()

and thats it.

running selenium tests from the command line

12 01 2008

I wrote a tutorial on how to run selenium tests from the command line. it is very handy you plan to include it on your continuous integration process.


selenium is great tool for writing tests to web applications. and with Selenium IDE, it is a breeze to write tests. here i will describe how to run your selenium tests from the command line (instead of manually starting the server, opening a browser, selecting the test file…). if you dont use selenium yet you should take a look at it first.

i will not go over the advantages over having regression tests… but to be effective it should be easy to run the tests. it is very boring to manually run the tests, so once in a while when i decide to run it i found out that some regression bugs appeared, and they were not easy to spot because they might got into the code a while ago. even worse is when somebody else on team introduced the bug ;)

complete article

firefox profiles & batch friendly options

10 01 2008

on my previous post i showed how to avoid gui windows popping up on your screen. now i am gonna show how to configure firefox to be less interactive (you dont want it to be interactive when you are executing background tasks.)

first you need to create a new profile. see how to manage profiles. you need a different profile so it wont mess with your default profile, and will allow two independent instances of firefox.

in order to have two simultaneous windows with two different profiles you need to run it using the following command line:

$ firefox -no-remote -P testProfile

if you dont give the -no-remote optional parameter it will ignore the profile parameter and will open a new tab on the currentelly open firefox (if you have one already open).

now that you have a new profile and know how to start it, you just need to configure some options to disable confirmation dialogs.

1) On the menu go to: Tools->Options…->Security->Warning Messages->Settings. Disable everything.

2) one very useful feature that you dont want in your test profile is crash recovery(or session restore). to get rid of it type ‘about:config’ on the location bar (a.k.a. where you type the url). set browser.sessionstore.resume_from_crash to false.

thats all. on the next post we will get some real action…

running gui apps on the background (invisible)

9 01 2008

here i will show how to execute an app that requires GUI (X window) to run on the background. i mean you wont be able to see the screen.

you may wonder why someone would want this since without the GUI you cant interact with the app. the test suite for the application that i am working on requires running some javascript test on firefox. i managed to write a script that automatically starts firefox run the tests and than kills the firefox process. but the problem is that while i run the tests i would like to keep working on another stuff but the windows keep popping up and bugging me.

this problem is simple to solve. i used a vncserver. a vnc (virtual networking computing) server will create a virtual X server. so you actually doesnt need to have a real display. usually, vncserver is used to display an application on a different machine. a remote machine would connect to the vncserver using a vncviewer…

you will need to install a vncserver. on ubuntu i am using the package vnc4server.

after installing, the first step is to create a password using ‘vncpasswd’. this password will be used to authenticate any vncviewer trying to connect to this server. we wont setup any viewer but vncserver requires this password anyway. so just type:

$ vncpasswd

now start the server

$ vncserver

if you dont specify a display number it will get one for you. it will probably be ‘:1’

our vncserver is up and running. to use it you need to change the env variable DISPLAY to ‘:1’ (or whatever display number you got).

now any process started on this shell will use this display. since it is a virtual display you wont see anything. just what we were looking for.

some applications also have a ‘display’ parameter, so you dont need to set the env variable.

$ firefox –display=:1

thats all.

Open Source and Software Estimation

8 01 2008

Software is hard , estimation is even harder (when possible ). There are many other articles on the subject. I am not qualified and will not try cover how to do it.

I will just share some of my insights on the subject from my experience in software development. Especially in the case of adopting open source projects within the project, and the unavoidable uncertainty that comes from using/relying on third part software.

Open source

I first understood the importance of open source at a time I barely knew anything about it. In my young arrogance, I was trying to build a simple 2D CAD system as side project on my free time. At that time, all my knowledge about graphics libraries was the canvas component from Delphi (lol). While debugging my application I hit a black box. The bug was not in my code but inside “that” compiled library that i could not look inside. I had no way to solve the problem. Maybe I was just using it on a wrong way but without looking inside that black box I could never figure out what was wrong…

I will not discuss if an organization should use open source or not. But me (as a developer) will always choose to work with open source.

I will use the terms “Black box” to describe a program where the user doesn’t see its inner workings. And “White box” where the user does see its inner workings.

White box

What i really like about the engineering software projects that I worked on during my college days was that there was almost no external dependency. Everything in C or C++ most of them dealing with mathematical models.

Starting a system from scratch, not bounded to any framework and free from learning how to use a library is great. Using a handy lib that do what you want is good. But programming a system that doesn’t use any lib is more fun. Of course that doing everything by yourself, what you can do is quite limited.

I was happy to work using an open source lib in my first real programming job. So my first task was something like this: this library can do “this” and we need “this” more “that” bit.

At first I did some trial and error. I mean, I was not really understanding what was going on “inside”. Soon it became clear to me that I wouldn’t achieve anything this way, so I dove into it. But I was completely lost. I was still in college and my biggest project had 4k lines of code, and this was much bigger. I also lack knowledge on the subject and as most open source projects the documentation was not part of its strength.

So I spent 10 (or 20) times longer understanding that huge amount of undocumented code than coding itself. Ok, the time I took to understand (a small fraction of it) is insignificant compared to the time I would take to write it (if I would be able to write it) so not that bad.

When you need to use something that will not act as a black box for you. You need time not only to learn how to use it. You also need time to understand how it works. This is obvious. I am not claiming I did a great discovery but a lot of people forget to take this into account.

So the first thing that needs to be estimated is how long it will take to learn X library or framework. Things are really hard to estimate here because most of the time you want learn just enough to execute your task. Important factors are size, quality of documentation and code. Before really reading the code and documentation how can you know its quality? So how can you even estimate the time you need to learn it?

And only after learning it you can have a reasonable idea of how long it will take to accomplish a certain task. In practice you will start doing your task and learning X framework at the same time. Often the result is a prototype.

Black Box

When you decide to use a library as a black box, user documentation gets even more important. Finding out if X library will suite all your needs is a really tricky task. Even if it has a good documentation it is very hard. But it is worst, some libraries claim doing stuff that they don’t really do, or just plan to do. And not all of them has the practice of documenting their bugs.

So a small task or bug using library X can easily turn from one hour to one week. The transition from a black box to a white box where you have to understand its internals is very time consuming.

Most of the time, assuming that you will need a white box understanding of all involved parts makes the project not feasible. The user base and stability of the library play a major role here. Also very important is how innovative is what you trying to do. If you are pushing the library to new limits you must probably will be the first to see some bugs. More on this later…

I guess the best you can do here is to assume that some black to white box will happen.


From wikipedia “A software framework is a re-usable design for a software system”. A framework is a result of extracting the common parts of a well known system pattern. They are great doing what they were planned to do. They give a huge boost on productivity, and you can do those amazing screen casts in “20 minutes”. But of course they don’t do magic. I guess to a non-developer they give an idea that everything can be done easily. So remember that whatever need to be done that your framework does not provide will take much longer than if you have a framework aid for the task.

Another great advantage of using frameworks is that they give you the basic design of the application. And it usually means a successful design. Whenever you build an application without relying on a framework, you need to design it. I guess no software developer would be arrogant enough to think that they can design something but trivial in the first attempt.

Most probably your basic design will need to be written and re-written sometimes while you learn about it, until you achieve a good enough design. So plan to write it more than once. The complexity of your system is the key point here, a good design is the result of planning and a lot of work. Who never heard the quote “System design is one percent inspiration, ninety-nine percent perspiration”?

Not exactly this

And the worst can happen. After working for a while with X you can find out that it is just not suitable for your requirements. And what happens if there is no Y to put in place X? You get one more project to develop, for FREE! I hope you didnt burn half your schedule to get at this point. Well, thats why checkpoints are there. In case like this is when you have to think again if developing the project is worth or not and make it clear to all stake-holders.

This mostly probably happens when you are developing something new, pushing the state-of-art of your domain, or just from the library you are using.


Every new X library/framework that you incorporate to your development stack is like a small R&D project. Research and Development are harder to estimate but it is easy to understand that they do consume more effort than a “normal” project.

Open source is great. It makes possible small teams to compete with big companies, but wait big companies also use open source… So don’t underestimate (too much – everybody always do anyway) development time, open source does not solve all your problems like magic.