Lianne’s Screencast

My Ups and Downs

So we’re coming up to the end of a turbulent four months. When it started out, I was building a new extension (Excel Exporter), with the primary purpose of testing out the extension support. About halfway through that, we realised there were some problems with the whole idea of the extension, and I was told to move on to something else. Next, I tested out the configurability of extensions, which as it turned out already worked. So there wasn’t much to do there. Finally, I was given webhooks.

My task: to take the old webhooks code, which was in the notifications area of Review Board, and move it into an extension. No big deal, I thought. After all, I’d already half built an extension at the start of the term. As it turns out, I didn’t understand the framework quite as well as I’d hoped. I got the code moved over fine, and was able to figure out where to put everything, so that was fine. But then there were errors, and I didn’t understand them.

The first few errors weren’t in the code, but were a result of needing to sync the database, or update the extensions branch. So there were some lessons learned there. After that, I realised my biggest mistake of all. I was treating this extension as a static thing, like the other extensions. I was acting like there was just a webhook, and users would set what url it would point to. As a result, I was never actually creating webhooks in the database, and that’s what the code depended on. So, after a lot of debugging with pdb, and a lot of help from Mike and Christian, I started actually creating webhooks, so a user could create multiple webhooks which all point to different scripts.

So now, it works! I wrote a php script that posts a message to Twitter. All it says is “Someone has posted a review or commented on a review!” but of course a user could make it say whatever they wanted. As of right now, it definitely works when a new review is posted, though I’m having some issues with comments.

There are some things that should be in the extension, that currently aren’t, due to running out of time. For one thing, webhooks should be configurable to associate with a specific repository and/or a specific group. Also, currently a webhook automatically listens to all types of events, while it would be better if the user could either select one, or select as many as they want from the list.

It’s possible that I may work on these things after the term is done. At this point it’s hard to be sure, since school has crushed my soul and I don’t even want to look at a computer for a while after exams are done. But I’m sure I’ll feel better after a couple of weeks of sleep and relaxation. In any case, before jumping back into Review Board, I’ll want to actually learn Python properly. Of course I had to learn a fair bit on the go during this course, but with a crazy schedule and just trying to keep up with everything, I never had time to do tutorials, start from the basics, as I prefer to do when learning a new language.

Hopefully I haven’t rambled on too much. It’s been a stressful term, and I’m relieved to be coming to the end. That being said, I’m glad to have the experience UCOSP has given me, and I had a lot of fun when I wasn’t panicking.

Screencast coming soon! It should be up tomorrow (Tuesday).

Status Updates – November 21st

Hongbin

• Refined the architecture of the RBBugTracker extension. All the work is the preparation for adding another bug tracker ‘BugZilla’.
• Added several page to let custom configure the extension.
• Did some manual test for GoogleCode bug tracker.
• Updated the code review and Github for this extension.
• Road Blocks: None.
• Next Step: Supported ‘BugZilla’.

Lianne

• Created new extension for webhooks.
• Created models.py within that extension.
• Road Blocks: None.
• Next Step: Finish moving webhooks code into the extension.

Brendan

• Fixed (hopefully all) the bugs in client.
• Mostly finished an rb-diff command.
• Setbacks: having a couple weird issues with a file not recognizing updates in an imported file, have been using a workaround thus far.
• Next Week: finish diff, + another few commands; create a config system for storing/updating basic information like server url, etc

Lindsey

• Made rb-upload which allows you to upload a screen-shot.  Tried to upload diffs but couldn’t get it working.  Also, made a basic graphical resource browser.
• Road Blocks: Uploading diffs.  Not too sure why this isn’t working as it seems I am sending the same data/format as does post-review.  Will have to continue to investigate this.
• Next Steps: Get uploading diffs to work!  Once this is done rb-create and rb-upload/patch can be made.
  Ask about making another API component which would read/write a config file for the rbtools API.

Laila

• Finished rough file attachment backend.
• Working on rough but functional UI.
• Road Blocks: None.
• Next Steps: finish a decent UI and see what people think; clean up code and add any minor features that are missing.

Kevin

• Status: Working on user page.
• Road Blocks: None.
• Next Steps: Finish page.

Goals for the Week

I started work on the webhooks extension this week, so these are my goals for the weeks to come:

21 November – 27 November

• Finish up writing the extension.
• Start on configuration of the extension.

28 November – 4 December

• Finish configuration of the extension.
• Fix any bugs.

5 December – 11 December

• Create screencast.
  

Minutes for the 14th of November

We went through our status updates:

• Brendan has finished up some more code and merged with Lindsey again. All they need to do now is write up more commands.
• Brendan and Lindsey need to watch out for changes made upstream.
• Hongbin’s computer died last week, causing him to have to rewrite some code.
• The Google Code part of his project is done, and now he’s moving on to other bug trackers.
• This link might be useful for Bugzilla stuff: https://wiki.mozilla.org/Bugzilla:REST_API
• Laila’s installer is pretty much done. She has posted a review of issues she’s found with the installer.
• She is starting to work on file attachment now.
• I (Lianne) have written this blog post about my current status and next steps.
• Lindsey has been working on things like basic authentication in the server interface, a RootResource class, and being able to add scripts and have them run through “rb <new script name>”.
• Lindsey and Brendan need to try to clean up their commits using git rebase -i <parent branch> to merge commits.
• Kevin has been implementing the user page, which displays your gravatar, groups, and reviews.

Since there are only 4ish weeks left in the term, we should be laying out our goals in a blog post each week. Also, we should be planning what we want to demo.

This week, our goals are due on Friday.

Webhooks

Yesterday, I spent some time on IRC with Mike figuring out this whole webhooks thing. Before that, all I knew was that webhooks are used to notify external services when something is updated. Mike explained a lot to me, so I thought I’d write about it here to help myself get it all laid out nicely.

I need to build a new extension. It must:

1. Register to listen to Review Board events.
2. When an event happens, turn the information into JSON or XML (I’m not sure which at this point.).
3. Find the appropriate webhook.
4. POST.

There are some things I need to consider, such as:

• There may need to be webhooks that are distinct between repositories within one instance of ReviewBoard.
• I need to determine how information will be serialized into JSON or XML.
• The extension should be designed for testability, so we should be able to modify it so instead of sending out a POST, it prints out something we can test.

Hopefully some of these issues will already have been decided on in the original webhooks code. Anything that hasn’t been, I’ll be deciding along the way.

So in the current webhooks code, a Webhooks class is defined in models.py. It has a dispatch method, which is where the payload is turned into JSON and the POST is sent. The actual POSTing is done in another file called post_url_dispatcher.py. In admin.py, there exists functionality allowing users to change the url for each webhook through the admin interface.

One problem we came across was that in the current code, publishing reviews, review requests, etc. was generalized into publishing. Mike and Christian both said that they disagree with this generalization, so I’ll have to figure out how to separate them when I write the extension.

The rest of the code is largely signal passing stuff, which I think I understand fairly well.

So basically, this is what I need to do next:

1. Create a skeleton extension in rb-extension-pack, like I did for my Excel exporter.
2. Create a webhook model in the extension.
3. Have the extension register for a signal.
4. Make sure I can configure the extension with webhooks to associate with various things. This has something to do with GenericForeignKeys. I’m a little fuzzy on what this part means at the moment…

For now, the extension should be dealing with the following types of signals:

• The publishing of review requests, reviews, and replies.
• Discarding/deleting/submitting review requests.
• Diff updates/publishing of drafts.

Also, these are the Django docs I should read if I need to further understand some things:

• http://docs.djangoproject.com/en/dev/topics/signals/
• http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/

As of right now, I think I’m understanding what I need to do pretty well, though questions will no doubt come up as I get deeper into the code. But overall I feel more on track now, thanks to Mike’s explanations and seemingly infinite patience.

Extension Configuration

Recently my task within Review Board has experienced a bit of upheaval. I found out that the extension I was working on didn’t really make a lot of sense. So, I was told to work on the configurability of extensions. No one was entirely sure if that part was working yet, so off I went into the extension code to find out.

I downloaded the rb-extension-pack from chipx86’s github, and set up the rbcia extension on my dev server. But when I tried to enable it, things exploded! Well, not literally. But there were errors, and the world of debugging python is not one in which I am well-travelled.

Luckily, Mike had seen these errors before, so I was saved from hours of slow-moving debugging. The problem was that the rbcia extension was out of date in regards to signal handling, so by commenting out any code that called the signal stuff I managed to get the errors to go away.

Next, I had to get the code of interest in the rbcia extension into my extension. I had no idea how many places I would have to copy code to get one page to appear! But eventually I had copied and adjusted all the code that had to do with configuration, and finally the config page appeared for the Excel Exporter! I made a test field, and tested that when I changed the value and clicked save, it really did save, and it did.

So now I have reached a point of some uncertainty. Is that all that needs to be tested, or is there more? I need to learn more about how exactly this configuration stuff is supposed to work. Then I’ll know where I’m headed next.

Status Updates

Brendan and Lindsey:

-Split up parts of the API and worked on them. The idea was to get a rough framework in place, and flush out some of our requirements at the same time.
-Came up with a standardized documentation scheme so that the API will be consistent
-Formed an action plan for dividing up work, and staying in communication.

Brendan:

-The parts of the API that I have been responsible thus far are: a ServerManager for authenticating clients (almost completely done), A Repository object responsible for storing information about the local system, and for performing tasks like diffs (part-way done, I know what I am doing though) and a utility class for various tasks like handling i/o, errors, etc. (written, but will be updated on an ongoing basis). My parts of the framework should be finished by tomorrow, at which point Lindsey and I can recombine code, flush code out, and remove abstractions.
-I plan on having the minutes from our first irc meeting complete and uploaded to the Google group either: before today’s meeting, or shortly thereafter.

-Problems: scheduling issues on my part. Resolved by creating a consistent and structured schedule for getting work done.

-Next steps: combining code with Lindsey and creating a functional (albeit bare-bones) API up and running.

Lindsey:

-Created the ServerInterface class to make RESTful HTTP calls to a ReviewBoard server.
-Started making the Resource class to handle resources.

-Problems: None.

-Next steps: Continue working on the Resource class, and put what I have done together with what Brendan has done.

Laila:
-Got installer to install python iff it’s not already installed.
-Set up the ui (more or less) to allow choosing optional installation components.

-Problems: None.

-Next steps: Install reviewboard and easy_install dependencies, and work out some ui bugs.

Kevin:

-Fixing invalid user/group bug.
-Implementing collapsible reviews.
-Contemplating implementing post-commit reviews.

-Problems: Review Board conventions (silly spaces) and insufficient knowledge of code.

-Next steps: Finish bug, finish collapsible reviews (soon), and start implementing post-commit reviews.

Lianne:

-Finished my first code review. 🙂
-Familiarized myself more with the extensions code.

-Problems: I need to set a concrete time for working on this to make sure I’ll get stuff done every week.

-Next steps: Continue working on Excel extension, fixing problems I find in the framework along the way.

Hongbin:

-Investigated the GoogleCode API.

-Problems: I need to investigate how to do a secure authentication by utilizing GoogleCode API.

-Next steps: I am going to write a proposal about reviewboard issue tracker integration next week.