Status Reports – November 16, 2013

Elaine:
What you’re currently working on, or have just completed:
I finally finished working out the POST and GET methods for my checklist.

What you’re working on next:.
I’m planning to have the PUT and DELETE done by tomorrow as well, and have a non-WIP request up by the end of the weekend.

What’s blocking you from progressing:.
None.

Any questions you might have.
None.

Allisa:
What you’re currently working on, or have just completed:
Finished cleaning up what I have and submitted it for a WIP review.

What you’re working on next:
Making all the changes from the review (there are quite a few, heh). Moving onto the next check (allowed hosts)

What’s blocking you from progressing:
None.

Any questions you might have:
How exactly would I format an htaccess file in terms of testing its execution? I’ve tried to look up examples of htaccess files but am not sure what’s needed.

Behzad:
What you’re currently working on, or have just completed:
I’m currently working on refactoring the code I’ve written.

What you’re working on next:
I will make the remove the hard-coding to allow for pluggable Trophy classes.

What’s blocking you from progressing:
Nothing.

Any questions you might have:
None.

Natasha:
What you’re currently working on, or have just completed:
I just pulled the latest changes from master and moved some of my changes around to get them working again.
I’m about to start working on updating the back end for suggesting reviewers again.

What you’re working on next:
I will continue updating the back end and hopefully post a review request for it soon.

What’s blocking you from progressing:
Nothing.

Any questions you might have:
None.

Edward:
What you’re currently working on, or have just completed:
I’m working on a prototype for indexed search using Haystack and Whoosh (https://reviews.reviewboard.org/r/4935/).

What you’re working on next:
Combining separate search fields into a single search box with query parsing.

What’s blocking you from progressing:
I’m having trouble finding a more powerful and suitable query parser to replace Haystack’s simple query parser with.

Any questions you might have:
None.

Meeting Minutes (November 3, 2013)

Announcements:

Consider how far you are in the project, how much you feel you have left to do. Consider how much time you need to spend in other courses, and how much time we’ll need to review your code. Factor all this in and course-correct if necessary. And if things are looking sour, let us know ASAP.

If you are blocked on reviews, come (gently) bug us.

Elaine M:

Is it better to have captured variables in the url, or pass them as parameters?

It depends on what you’re doing. For RESTful stuff, you pass the parameters for the things you want in the URL for a GET, where you are trying to retrieve information.

Think of a URL as a fixed location. Any identifying information that represents that location should be in the URL. Optional values, for, say, filtering lists, would be query parameters.

So if I create a new checklist for request 103, it should be something like: checklist/new/103, rather than checklist/new/request=103?

No, you never decide on ID’s and URLs shouldn’t imply actions. That’s what HTTP methods are for. So to create something, you do a POST to a list URL. So POST checklist/ with a payload for whatever data you want to use for the creation. The resulting payload from the server will represent that checklist, and as part of that, you will get a link to where it lives (like checklists/123/)

Is it only with GET where I actually have to use parameters?

A good resource to look at would be webapi/resources/default_reviewer.py. Don’t think of them as parameters. Think of them as addresses, locations.

Note that our WebAPI resource code makes doing this pretty simple – it takes care of a lot of the boilerplate for you. Checklists/123/ is a fixed address for that specific instance of the resource. It could even be checklists/my-name/, if the WebAPIResource class said that was the address. Though database-backed IDs are usually best.

WebAPI stuff can be pretty tricky, but we’ve also done quite a bit of work with it recently, so a lot of webapi stuff is swapped into our brains. Ask us questions when you’re stuck. Just follow the pattern that default_reviewer.py provides for get_queryset, create, and update, and it’ll do most of the rest.

Natasha:

I just posted my first front-end review request. At one place I’m just repeating stuff that I had put into a function because I couldn’t figure out how to call the function form the right scope.

We will do a review pass today. If you’re way off rails, I’ll let you know.

I haven’t gotten my Mac dev environment set up.

We will sort out your dev environment stuff right after the meeting. Basically, we’re probably gonna point you at the vagrant stuff which is pretty straight-forward.

Edward:

I noticed r/4662 is marked as submitted, but there are still some outstanding issues, so I’m a bit confused.

It seems like someone closed it by accident, feel free to re-open it.

I’d like to add an option to rbt post so that it automatically opens the URL after a review request is posted. Just want to get some feedback on this idea. I’m still looking into other project ideas but I think this is a quick and useful one.

It already exists: -o

If you’re looking for additional projects, I think prototyping out a new search implementation would be pretty sweet. It would make a major difference to people.

It’s probably too big to rewrite search, but a prototype that built up some knowledge of how Django-haystack works would be awesome.

It’s mostly backend – it’d be indexing. I imagine the search UI would be more or less the same. Possibly some different things in the auto-suggest.

Maybe start checking out haystack and get familiar with it. And start coming up with ideas on how we could prototype its usage in RB.

Sounds like a good idea. What files should I start looking into for the current search implementation?

reviewboard/reviews/management/commands/index.py is the existing lucene-based indexing code and ReviewsSearchView in reviewboard/reviews/views.py is what does the front-end search. There’s no dynamic stuff for search right now, it’s just a GET with a query string that returns a page of results.

Haystack is not currently used in RB. The existing search was written before haystack existed and requires that people compile pylucene, which is a huge mess of java. I think we’d like a new implementation to use haystack with the whoosh backend (at least by default), so that it’s all pure python. Most people don’t set up search because of all that and when they do, it often doesn’t work, because (py)lucene is bad at backwards compatibility.

Behzad:

I got most of my questions answered yesterday. I have a few more though: do we want to allow for a review request to be able to have more than one trophy?

I think it’s going to be inevitable as we create more trophy types that there’ll be an overlap.

I’m not sure how to put of new iterations of my code in the same review request.

rbt post takes an -r argument for which you can pass an ID. That’ll upload your current branch vs. master diff as a review request draft. Look it over, make sure you dig (and re-post if you need to make fixes).

Alissa:

Would it be ok to do a blog post about web vulnerabilities?

That would be great.

Are there any examples in the application of files being uploaded/downloaded?

Django has a concept of a FileStorage, which FileField uses for uploading. You can write to a path using it, kind of like standard file I/O. You should be able to write to $MEDIA_ROOT/uploaded/files/, maybe in a _security directory or something.

For downloading, there are two ways to do this:

1. The Python code can open a request to the server itself is on and fetch the files (allowing all the logic to remain in the Python part of this).

2. It can emit JavaScript that makes the attempt (success/fail reporting has to be a little more controlled on the JavaScript side).

The first option is probably simpler and nicer. It does mean that certain browser quirks (like IE’s attempts to ignore mimetypes and sniff contents) won’t be checked. Even then it would only be checked if the admin is running IE, which seems less likely. So #1 is fine.

Choosing Vagrant to Set Up my Development Environment

When I was at the phase of setting up my development environment for Review Board, I wasn’t sure in which direction to proceed.

I am running OS X, and so I wouldn’t need to set up a brand new Linux virtual machine for development purposes as Windows users would, but I had been told that there are some dependency issues with some versions of Linux and OS X.

I decided to go with the second option of using Vagrant. Vagrant is used to build and deploy virtual machines. It is mainly used for the purpose of having everyone on the same development team use the exact same operating system to avoid dependency issues.

Steven McLeod configured Vagrant, Ubuntu 12.04 and the Review Board project. The configuration repository and installation details are located here.

We were given licenses for Vagrant VMware plugin and VMware Fusion or Workstation depending on our current operating system.

I had had experience using VMware Fusion with Windows 8 before, but still found it hard to grasp what exactly Vagrant was doing.

I used to open up VMware Fusion and start the Windows 8 VM. With Vagrant, I accessed the VM through Terminal (the command-line based terminal emulator included with OS X). I wasn’t sure in what way it was using VMware Fusion, or exactly what was going on. After asking questions from the mentors, things became much clearer.

In Terminal, from the repository’s directory on the host OS, the command “vagrant up” starts the VM using VMware Fusion. From that point, the command “vagrant ssh” SSH’s into the VM and you can begin to develop, run the server, or run unit tests. You don’t have to manually start VMware Fusion.

I wanted to use a GUI based IDE to develop code, and could not do this by SSH’ing into the VM. Luckily, Vagrant allows the repository to be shared between the host OS and the virtual machine. I downloaded and installed PyCharm on OS X (whose configuration details with Review Board can be found here). I now develop on OS X, and run the server and unit tests by SSH’ing into the VM 🙂

Status Reports – October 12, 2013

Behzad Raeisifard

What you’re currently working on, or have just completed:
I am working on creating the TrophyManager model.

What you’re working on next:
I’m going to write unit tests for it.

What’s blocking you from progressing:
It’s not really blocking my progress, but it would be helpful to have my question answered.

Any questions you might have:
I’m not sure in which file the TrophyManager model should be.

Allisa Schmidt

What you’re currently working on, or have just completed:
I’m working on implementing a basic security test framework.

What you’re working on next:
Going to post it up for a WIP review shortly.

What’s blocking you from progressing:
Nothing at the moment – I talked to Christian about where I have to go next so I have a pretty clear idea.

Any questions you might have:
None.

Natasha Dalal

What you’re currently working on, or have just completed:
Submitted a patch for the fill-database command. Completed a basic back-end function that returns a list of top 5 reviewers who are likely relevant to a review request by looking at comment history… It is horribly slow and inefficient.

What you’re working on next:
I’m going to be reading about how to do things more efficiently in django so I’m not querying the database an unnecessary number of times. Also maybe later today I will be doing some front end things to get the reviewer names to show up on the review request page, and make them selectable.

What’s blocking you from progressing:
Nothing.

Any questions you might have:
I was reading some existing code and came across queries such as “q = DiffSet.objects.filter(files__comments__review=review)” I’m not completely sure I understand what’s going on (I’m assuming each of the words in the parenthesis corresponds to a related_name field? )

Edward Lee

What you’re currently working on, or have just completed:
I am working on a new ‘rbt post’ to option automatically determine the existing review request to update.

What you’re working on next:
Improving the detection algorithm for the new ‘rbt post’ option

What’s blocking you from progressing
None.

Any questions you might have:
Not a question, but I’d like some reviews on my requests to help me know how I’m progressing.

Mary Elaine Malit

What you’re currently working on, or have just completed:
I created the checklist extension, and I’m trying to hook the template into the Review UI.

What you’re working on next:
Get the extension to show up in the admin list, and my template rendered into the UI.

What’s blocking you from progressing:
I don’t know why my extension is not detected by RB.

Any question you might have:
I don’t have any specific questions, because I’m not exactly sure what’s going wrong. Any tips on how to get my extension recognized and hooked into RB would be great.

 

Meeting Minutes (September 29, 2013)

General:

Karen Reid sent a message asking you to make short videos about yourselves talking about your sprint experience. You should do it ASAP while the sprint is fresh in your minds.

If any of you, at any time, are blocked entirely on code review, come talk to us, ASAP. One of our primary purposes as mentors is not just to answer questions, but clear things out of the way. Gentle pestering is the recommended approach.

You’re not prevented from writing blog posts outside of your scheduled day. You can blog every week if you want.

Allisa:

For making time estimates for our work, do we make up tasks we think we have to do then estimate time for that task alone?

More or less. You can have something like

“Create empty pages in the admin for security stuff: 4 hours”. A plan is better than no plan. It is entirely realistic to have your time estimates will be off. A rule of thumb we often used was:

1)   Figure out how long you think it might be

2)   Factor in time for reviews, questions, getting stuck, needing to make requested fixes (this will all be a guess)

3)   Double that 🙂

Breaking down your project into subtasks is a good exercise and it’ll give you a sense of scale of things and a map to follow.

Where are we posting our tasks/estimates?

The blog (https://reviewboardstudents.wordpress.com/ ) or hackpad (https://reviewboard.hackpad.com/).

How much detail is expected for the mockups?

We are looking for quick sketches, nothing pixel-perfect. Wireframes, drawings in a notepad are fine. If we need more detail, we’ll ask. UI-heavy things like the checklist extension probably require more thought.

Elaine:

Do we have to post the mockups for a review request?

I recommend it. It’s a great way of bolstering a review request, and clearing up what a review request is proposing. It’s especially useful if you are looking for feedback on your designs. Everyone can post “just screenshots” as review requests. You can do so by going to https://reviews.reviewboard.org/r/new/ and at the top of the left column choosing “None – File attachments only”

There is no distinct formal process in place but posting screenshots to Review Board has distinct advantages, the main one being that comments can be added to regions of the screenshot.

Either way, the goal is to show what you’re thinking, so we can give a better feedback.

Down the line, when you have patches that make changes, it’d be good to attach screenshots showing what the patch is actually doing. When writing a spec it’s always good to have your mockups on the hackpad for the spec. Posting them on Review Board to get feedback is fine as well. It’s up to you how you want to do this. It’s your project.

Behzad:

In https://reviewboardstudents.wordpress.com/calendar/ soft pencils are down is on Dec 1 and hard pencils are down is on Dec 3. I remember something being mentioned about this at the sprint. Could you explain it again please?

“Pencil’s Down” is when your project should be in a state where they’re either ready to merge, or (if you didn’t get as far as you originally thought) ready to be handed off to someone else. In either case, they need to have gone through some review at that point.

Since reviews can take sometime, especially with 5 students and large projects, so you should budget in time for that.

To be on the super-safe side, have the “final” version of your code out of WIP and ready for serious review somewhere around a week before firm pencil’s down. We need to have gone through your code, tested it, and shepherded it into a good state if you want a chance at a decent evaluation.

So soft pencil’s down is kind of like a warning. If you’ve hit that point and no code is being reviewed, that’s not good. Hard pencil’s down is the 100% serious cut-off point.

Hard-pencils down is the very last chance to submit code review if you want it factored into your evaluation.

We might do a review, then you fix stuff up, we do another review and keep iterating like that. We will be doing that until pencil’s down.

Pencil’s down is the last day for that iteration to occur, and we should be at a pretty stable point. Like, if on or near Pencil’s down, we go “This needs a architectural overhaul”, there’s been a serious failure on our part. It should mostly be little fix-ups.

So you don’t review individual task completion so much as the project as a whole?

That depends on the how you break up the project. For instance, you might write the foundation for something that can do security checks, without implementing all the checks. That can be reviewed and go in. Then you could implement a set of checks, and that can go in.

We can (and want to) do WIP patch reviews all the way up to the end of your project.

In the calendar, some of the Tuesdays are for some of us to post blogs. What should the blogs be about?

UCOSP requires you to do some writing as part of this course. What we ask is that you write something about your experience working on this project that’s suitable for a technical audience that might not be 100% familiar with the internals of Review Board. You might talk about a tool you’ve been using or a bug you’ve had to fix, and how you went about figuring it out.

UCOSP uses this is two ways:

1)   To get a writing sample which they can evaluate as part of your final grade recommendation

2)   To post on the UCOSP blog to show prospective projects, mentors, sponsors and universities to show that this stuff does work and that you do learn.

If you need inspiration, or aren’t sure what kind of material to write about, you can just look at previous student posts.

There is no activity shown on my github account. Is there a reason for that?

I believe it takes a little while. I think github handles these things asynchronously for the first commit, so it won’t know you committed until it has swept that project, which it only does every once in a while. But after the first one, it knows you’ve committed there, so it gets updated automatically.

Natasha:

If a task is going to take me 4 hours to code… and then I want to factor in review time, but there may be like a week of lag time between when i post it and when it’s reviewed, am I now estimating a week?

No, unless that task is blocking you from working on something else.

Since you’re using git, you should be able to continue your work (generally on a new branch off the branch for the change that’s up for review).

Realistically, do students generally complete just the one project during the semester?

It’s really hard to make a general statement about that as it varies wildly. Every student is different, every project is different. Some students work on one project the whole semester while others are able to knock off a few projects. In some rate cases some students completed the project after the semester was complete (in which case, it was known beforehand).  It’s really dependent on the student and the project. Sometimes these projects hit into problem areas we didn’t really intend.

It is not a race to see who can finish the most projects.

Schedule your project realistically. If it seems like it shouldn’t take up the whole semester, plan to have a second one. Try not to artificially make that projects take the whole semester.

I have questions about the requirements for the rbtools command (for creating .reviewboardrc files) I’m working on. The code I’ve written so far takes in options for the server URL, repository, and branch, and creates a .reviewboardrc in the cwd. I could use some direction as to where to go from here.

Having it ask questions would be good, if you don’t pass options. Along with that, I’d love some options to make it easy to create a .reviewboardrc for RBCommonds.com. I can help you with what’s needed for that.

The RB URL will need to be manually entered. The repository name should optimistically be determined, based on what repositories are known on that server and the current checkout’s settings.

If it can’t be auto-determined, or if there are multiple options, it should ask.

RBCommons is out hosting service for Review Board installs for small businesses, classes, etc. http://rbcommons.com/

Ignore TREES, we we’ll probably nuke it by version 1.0.

I’m not sure if my blog post is up to par. I’m the first one to write one so I didn’t have as much stuff to blog about.

I think it’s pretty good. I actually expected a post about your sprint experience. But it definitely serves the purpose of helping UCOSP determine your writing abilities. It’s also good data to have on the blog the next time someone is setting up PyCharm.