Tracking Down the Origin of a Bug

When something breaks, or you find a bug, it can be helpful to identify the commit which introduced the faulty code. Git comes with the ‘bisect’ command, which makes searching for that commit efficient.

Bisect helps automate a binary search through a series of commits, to find the earliest one which exhibits the problematic behaviour. After providing an initial ‘good’ commit without the bug, and a ‘bad’ commit which contains it, bisect will iteratively guess a commit and automatically check it out for you. After you test this checkout and identify if it is ‘good’ or ‘bad’, Git will guess the next commit, until it has identified the first ‘bad’ commit.

Here is an example of how to use bisect:

$ git bisect start
$ git bisect good [commit] # [commit] is the sha1/branch of a 'good' commit
$ git bisect bad  [commit] # [commit] is the sha1/branch of a 'bad' commit

Git will then checkout its guess, and output the commit information. You should then test the code, and indicate to Git if the commit contains the fault using:

$ git bisect good # OR
$ git bisect bad

After a number of iterations Git will output information on the offending commit, ex:

b047b02ea83310a70fd603dc8cd7a6cd13d15c04 is first bad commit
commit b047b02ea83310a70fd603dc8cd7a6cd13d15c04
Author: PJ Hyett
Date:   Tue Jan 27 14:48:32 2009 -0800

    secure this thing

:040000 040000 40ee3e7821b895e52c1695092db9bdc4c61d1730
f24d3c6ebcfc639b1a3814550e62d60b8e68a8e4 M  config

You should then run the following command to restore your repository to its state before you ran git bisect:

$ git bisect reset

This post is based off of the following:
http://book.git-scm.com/5_finding_issues_-_git_bisect.html
http://progit.org/book/ch6-5.html

Out of Sync Commits

Sometimes, when doing a post-review, other people’s commits show up in your diff, this is due to out-of-sync branches.

This is a common issue but the fix is fairly simple.

From the terminal inside your review board root directory, type the following commands:

$ git checkout master
$ git fetch origin
$ git fetch origin master
$ git pull origin master
$ git checkout "feature branch"
$ git merge master
$ post-review -r "review id"

 
Where “feature branch” is the name of your out-of-sync branch, and “review id” is the id of your posted review.

That’s it, your branches should now be in-sync.

Yazan Medanat

Python Virtual Environments

Sometimes, one Python environment may not be enough. For instance, you may find yourself working on multiple Django projects at once – each with different Python package requirements. I personally ran into this problem when writing a feature which required the bleeding-edge version of Django. Repeatedly installing/uninstalling different Django versions is redundant, and virtual environments are the answer.

To get started, install virtualenv and virtualenvwrapper:
pip install virtualenv
pip install virtualenvwrapper

Add the following to ~/.bashrc:
# virtualenvwrapper
export WORKON_HOME=~/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true

Lastly, create the virtual environment directory and execute ~/.bashrc:
mkdir ~/.virtualenvs
source ~/.bashrc

Now you can create isolated Python environments using mkvirtualenv envname. I have one called “rb”, and one called “rbalpha” which contain different Python packages. After an environment is activated, you’ll see (envname) next to your username in bash. You can activate an environment by simply using workon envname.

Once you are in your environment, Python packages can be installed in a similar manner as before. The only difference is you don’t use sudo when installing packages with pip or setup.py install. If you do this, the packages will be installed to the system’s Python environment.

Pro tip: To list the current environment’s Python packages, use pip freeze.

The Python Debugger (pdb)

To break into pdb from a running program, insert this right into your code:
import pdb; pdb.set_trace()

You will then be in the debugger prompt, where you can move up/down in your code (u/d), look at variables (p expression), or execute any Python syntax in the current context (eg. changing a variable). A list of debugger commands is here.

To print an object’s complete state, the following can be used:
pp vars(your_object)

Running Subsets of Review Board Tests

Have you been writing tests for Review Board?  If so, you’ll want to read this.  This is copy-paste from a super-useful e-mail that David Trowbridge sent out to the UCOSP mailing list:

Mark in our UCOSP group this semester had a question, and I think the answer would be handy to many of you who want to use our test suite. Specifically, because the suite takes several minutes to run, is there a way to run a subset of the tests?

We use a test runner called “nose”. Presumably when you were running through the initial development instructions, you installed this package. It supports a lot of special arguments for limiting test cases and other functionality, but any arguments you want to pass to it have to come after a “–” argument to manage.py test.

To run only the tests in a specific module:

./reviewboard/manage.py test -- reviewboard.scmtools.tests

To run the tests in a specific class:

./reviewboard/manage.py test -- reviewboard.scmtools.tests:GitTests

And finally, if you want to run only a specific test case:

./reviewboard/manage.py test -- reviewboard.scmtools.tests:GitTests.testFilemodeWithFollowingDiff

Some other flags which come in handy are -x, –pdb and –pdb-failures.

This will stop the test runner after the first encountered failure.

./reviewboard/manage.py test -- -x

This will drop into pdb (the python debugger shell) when an error occurs. In django test parlance, errors are problems that aren’t “test failures” per se — things like uncaught exceptions.

./reviewboard/manage.py test -- --pdb

Similarly, –pdb-failures will drop into pdb when a test case assertion fails.

Hope this help cut down your turnaround time!

Easy installation of older python versions in Gentoo

Sometimes when you are to provide backward compatibility with particular version of somewhat you’d better have at least two versions of it: the oldest and the newest ones.

RBTools have its developers to look back on Python 2.4.  Unfortunately, until recently, I had to manually check backward compatibility of used libraries and language features and sometimes failed to do that forcing reviewers to do extra work and reject my review requests.

The most trivial (but not easiest) solution is to fetch python sources somewhere in $HOME and build it manually:

$>wget http://www.python.org/ftp/python/2.4.6/Python-2.4.6.tar.bz2
$>tar xjf  Python-2.4.6.tar.bz2 && rm Python-2.4.6.tar.bz2
$>cd Python-2.4.6
$>./configure && make

I didn’t mention before that another useful thing for python developers is virtualenv which allows to have separate python environments. So if you already use it you’ll have a problem at this point: virtualenv does not allow to use specified interpreter unless it’s located under /usr/local. So you’ll have to move it at least there. As I’m one of those who tries to maintain their system as clean as possible I started to search another way to solve the problem. And suddenly I recalled that I’m running Gentoo with its complicated (at least for me) notion of package slots. Briefly speaking, slot system allows you to install particular piece of software and protect it from being removed or updated. To install package in a new slot you must providie ‘-1′ or ‘–oneshot’ option for emerge. The only problem remains – how to install some very old version? Easy! Just provide required version separated from the package name with a colon:

$>emerge -1av python:2.4

Ok, now you get it. From this point you just need to run virtualenv with specified python version:

$>virtualenv .ve-2.4 -p python2.4

This is it! It remains to repeat installation of required modules for the new environment. Optionally, I like to link frequently used binaries:

$>ln -s .ve-2.4/bin/python2.4
$>ln -s .ve-2.4/bin/nosetest-2.4

Developing with CSS Media Queries

My GSoC project is to develop a better admin UI for Review Board. I wanted to share my research and development on this blog. One of the challenges of developing a layout for the web is the variety of different devices with different resolutions.

I chose to use media queries to create a fluid layout for this dashboard.
Here is a simple example of a CSS media query:

@media screen and (max-width: 480px) {
/* CSS here */
}

It can also be used to link whole stylesheets:

<link rel="stylesheet" href="style/screen.css" type="text/css" 
media="screen and (max-width: 480px)" />

Installing Extensions

So for the last few hours of Sprint Day 2, I started trying to get extensions installed and working on my computer. The process was pretty complicated, and I am still not finished, but this is what I have done so far.

To start I had to read some material about python eggs, as a review board extension is a python egg.

Switch to the extensions branch of Djblets and Review Board

1)      Go to djblets directory and do “git checkout extensions”

2)      “git branch extensions origin/extensions”

3)      Go to review board folder and “git checkout origin/extensions”

Install the new djblets and sync Database

4)      “sudo ./setup.py develop”

5)      Make directory /reviewboard/reviewboard/htdocs/media/ext

6)      “./reviewboard/manage.py syncdb”

Execute Evolutions

7)      “./reviewboard/manage.py evolve –execute”

Add Extensions

8)      go to rb-extension-pack, and cd into rbreports

9)      “python setup.py develop”

After this, everything should have worked, but I ran into some issues when trying to load up my localhost Review Board page. The solution was to do the following:

10)  “git pull origin extensions”

11)  “cd /usr/local/lib/python2.6/dist-packages”

12)  “sudo rm -rf Djblets-0.6.7-py2.6.egg”

13)  go to  Djblets folder

14)  “sudo python setup.py develop”

15)  “./contrib/internal/prepare-dev.py” (in Review Board)

This enabled extensions, but I have many errors in my terminal. Still have to go through those!

Mark’s 16-step Guide to using Git

1.on branch master

2.git checkout -b issue123

3. do work

4.git add <file>… (only what you want to keep)

5. git commit -m “fixed issue123″

6.git push githubUsername issue123

7.post-review

8.mentor says to change something

9.repeat 3-8 as needed

10.review is approved

B. Bring a branch up to date assuming branch has been PUSHED

11.git checkout master

12. git pull origin master

13. git checkout issue123

14. git checkout -b commit_me_issue_123

15. git rebase master

16. git push githubUsername commit_me_issue_123

Using the Debugger

Try adding the condition statement chris suggested (form.groups.choices) which should allow a check to see if there is any info in the groups. But this didn’t work.  Mike then helped me step through the debugger.

check that python debugger is installed

in python: import pdb

Insert debugging code into the django code

import pdb;pdb.set_trace()

Run the debugger

Run the webpage where the source is located, then open up the server terminal where you can query the debugger commands such as “form.groups” to see the value of those fields.

Continue the debugger

pressing ‘c’ lets the continue progress past the break point.  If stdin is removed from the terminal access it can be regained by: ctrl+c to kill server, and ‘reset’ to start.

———————————–

Discovered that this field isn’t generated when there is no information form the person (ie: name, etc.). Will attempt to track it down with the debugger.