Lindsey & Brendan’s Screencast

Screencast Roadblocks and Solutions

There have been a number of problems, and some interesting workarounds in order for Lindsey and I to create a screencast. Our requirements were as follows: One of us needs to run through using a number of programs (web browser, shell running our scripts, and our graphical api tool). These actions are going to be recorded by screencast software, while we talked over top. Both people need to be able to view the contents of the screen during the recording.

The recording has to be done on Lindsey’s Linux box, as I cannot get his graphical tool to run on my computer (It is worth mentioning that the tool works fine on Lindsey’s computer, and is not one of our deliverables). One of the required components gtk would not install properly on my system. I need to install it through a program called macports that deals with linux->mac ports. The program had at least 100 dependencies that all needed to be fetched, extracted, verified and installed. On at least 10 occasions, one of those dependencies failed and had to be manually installed. Once the program finally ‘installed’, it still didn’t seam to work. So, I found another installer for a comparable gtk tool that could be installed via a shell script. This also didn’t yield any results.

Next in need of replacing was our original screencast software, a web-based program called Screenr (which is mainly designed for screencasts to share on Twitter). Unfortunately, the program does not run on Lindsey’s Linux machine. At the same time, Lindsey was having some issues with setting up a VNC Server to share his desktop.

We replaced VNC with Skype, as it can do screen-sharing (it works quite well), and Screenr with a linux utility called gtk-recordmydesktop. This introduced the next problem, getting recordmydesktop to record both Lindsey’s mic, and my audio. After many attempts involving configuring settings, all manner of hacked together cables, and a program called JACK (which probably could solve the problem, but is a complex program) we decided the only two real options were cranking up Lindsey’s speakers and recording my audio from the speakers (much like how NASA’s moon-landing video was shot by pointing a camera at a television), or to have my audio overdubbed. dubbing at this point seemed like the best option.

We wanted to be able to shoot the video in segments, and combine + crop the segments together. Youtube could handle combining and cropping, but not the dubbing (it seems), so this meant we needed to use another program for overdubbing. I own iMovie, so it seemed like the best bet. The only issue is that iMovie is really picky about the files types it can load.

recordmydesktop creates ogv files. Lindsey found a tool to convert the files into AVIs (or more correctly ISOs containing AVIs. iMovie kind of pretends it can load AVIs, but it is lying. Luckily, in the meantime, I had found a program called Handbrake, which can convert AVIs into MP4s. These MP4s can then be loaded into iMovie to be spliced together.

The overall system works as follows:

• Lindsey runs the various programs to be shown in the screencast
  • The programs are recorded by gtk-recordmydesktop and saved as OGVs
  • A stream of Lindsey’s actions are sent to me via Skype’s screen-sharing tool.
  • my audio is recorded (probably using quicktime, maybe Garageband)
• Lindsey converts the OGV to an AVI in an ISO using devede
• Lindsey creates a tarball of all the AVIs using gunzip
  • The tarball is sent to me using Yousendit, Skype or Dropbox
• I unzip the tarballs (using Apple’s archive utility)
• I convert the AVIs to MP4s using HandBrake
• I load the MP4s and my audio into iMovie and splice everything together
• The final result is loaded onto YouTube.

In all ~10 programs are used (not even counting the programs we are actually screencasting about. We are going to record the screencast tomorrow, and I will do the editing Friday (I don’t think I can do it before then, as I have a big exam Friday morning).

Who knew that making a screencast would have so many problems?

Brendan

Final Week

This really is the last week for any real changes to be made, as next week will be centered on making our Screencast, getting ready to commit our branch, and handling any last-minute problems that arise as the result of the ReviewRequest.

Lindsey and I have gotten a lot done (Lindsey is a coding machine), and I think we really are on schedule, so there are only a few things left that need to be finished up.

On Lindsey’s end, he has a change so that rb-* commands so that they can be called from anywhere, a minor adjustment to one of Resource’s subclasses, and work on the GUI client that we plan on using for our Screencast (Maybe some other things too).

On my end, I am going to be creating an rb-patch command, that will download a patch from the server, apply it, and optionally commit it. I am also going to be making some adjustments to how our configuration file is handled, so that it allows comments and grouping of variables. I also need to look into good software for making screencasts.

It has been an interesting project to work on. I have gotten to know about ReviewBoard, which is truly a well-thought-out piece of software. I am recommending that my current employer start using it. I’ve also learned how to use git (git mergetool is a lifesaver, and I’m unsure why it is not prominently mentioned to git newbies), and a much better understanding of python (I had been using python for writing quick scripts for a while now, but had never dealt with objects, or other complex interactions). Thanks to this project I feel like I could handle creating a real program in python.

Brendan

Meeting Minutes – Nov 28th

For the most part everyone seems on track – which is good considering all the projects due at this time of year

Brendan was wondering how to download a diff from a review board server.  Pointed in the direction of http://www.reviewboard.org/docs/manual/1.5/webapi/2.0/resources/diff/, specifically setting the mime-accept type.

rb-patch should: download a diff, and apply (merge) it to the current working copy.  It should optionally commit the change as well.

We should be installing all the rb-* scripts and have rb look for them to run.  Either put them in the path, or maybe the better option would be to put them in a special directory so as not to clutter the path.

Regarding Bugzilla and REST: it seems that the REST API is built on top of XML-RPC API so it would seem best to use XML-RPC API instead.  That being said, there isn’t enough time left.  Hongbin will focus on polishing what he has with REST API and getting his demo ready.

For Laila, regarding keeping some views public, in this case its ok to do so and import them.
Also, she’s having some troubles with captions in screenshots.  It sounds like its a bug though;  need to take a look at the template as it may be broken.
She was wondering if it was reasonable to not allow the editing of filenames – which is fine.
Also should files have descriptions (as opposed to captions, which are a little shorter.)  Again, that sounds fine as long as they don’t get *too* long (like more than one line.)
Finally, although its a good idea to have uploading files and captions in the same UI dialog, for now we should use two separate ones so as not to add any confusion.

Just a reminder, you do need to submit a screencast of your work.  The idea is to cover as much of what you did but still keep it short (don’t drone on!)

RB Python API Weekly Goals

Our plans for the following 3ish weeks are:

Nov 18 – Nov 26:  Make “create review request”, “patch”, “upload diff” and “get diff” commands.

Nov 26 – Dec 3:  Finishing touches to things and polish up the Graphical Resource Browser.

Dec 3 – Dec 7:  Clean commit history and merge with upstream rbtools.  Make screen-cast to showcase what we have done.

~Brendan & Lindsey

Uploading Diff Troubles

So the last two days I’ve been trying to be able to upload a diff through the API to no success :\  Interestingly though, I have been able to successfully upload screen-shots.

Also, I’ve thrown together the start of a Graphical Resource Browser.  Obviously there are lots of issues (I can’t seem to figure out sizing things in pygtk!) but regardless it’s something to work off of.

Resource Browser - Repositories List

~lindsey

Quick Update

I finally got all the changes done (or at least I think I caught everything) from Christian and Mike’s review regarding the serverinterface and resource modules.  Wooo!

Now working on putting together some client commands.  I’m hoping to be updating our review request tonight.

~lindsey

Status Updates Halloween

Brendan:

• This Week:
  • Started making changes to API based on feedback. Changes are ongoing
  • New code is up for review
• Next Step:
  • Finish current changes + new ones raised by new review
• Roadblocks
  • None

Hongbin:

• Status
  • Successfully implemented authentication to GoogleCode bug tracker.
  • Fixed a bug about the script “devserver.py” failed on Windows.
  • Discovered a possible bug of missing database’s name in generated script “settings_local.py”.
• Next Step
  • Implement the status notification to GoogleCode bug tracker.
  • Support other bug trackers.
• Roadblocks
  • None

Kevin:

• Status:
  • Posted first draft of ‘one-click shipit’
  • Posted third draft of ‘collapsible reviews’
  • Started planning the ‘user’ page
• Next Steps
  • Finish the mockups and implement the ‘user’ page
• Roadblocks
  • none

Laila:

• This Week:
  • Fixed many silent install issues: we now install everything but some python modules (using distutil) silently.
  • Fixed issues with changing the system PATH variable
  • Testing: the installer makes Reviewboard work (almost) out of the box for MySQL and Apache, using Mod_Python. fastcgi and wsgi require more configuration.
  • Lots of error-checking work (did an installation fail, is a Python module already installed…) and cleaning up the code
  • The installer now installs MySQLdb python module to work with MySQL, and pysvn to work with SVN.
• Next Steps:
  • Work on Perforce, Git, CVS, PostgreSQL, and Mercurial support
  • More testing of all these configurations and Python 2.6 vs. 2.5.
• Roadblocks:
  • None

Lianne:

• Status: Unfortunately not much has changed since last week, thanks to midterm season.
• Next Step: Move webhooks code into a new extension.
• Roadblocks: None, besides midterms.

Lindsey:

• Status:
  • Plugging away.  I made some large changes to our code structure based off the feedback given.
    The latest changes have been updated to the review request.
• Roadblocks:
  • Still not 100% sure how to search for children through the tree, when the child resource is based off of different field_ids then that of its parent.  For example:The “screenshot_comments” resource list has the following uri template:
    “http:\/\/demo.reviewboard.org\/api\/review-requests\/{review_request_id}\/screenshots\/{screenshot_id}\/screenshot-comments\/”
  • However, it’s child resource object, “screenshot_comment”, has a different structure of template:
  • “http:\/\/demo.reviewboard.org\/api\/review-requests\/{review_request_id}\/reviews\/{review_id}\/screenshot-comments\/{comment_id}\/”
• Next Steps:
  • Wait for feedback from the latest post, and while doing so I’m going to try and work out the issue I have come across.

Minutes for October 24

Reminders:

• get your status reports in to whoever is compiling them
  • do so in a timely manner

• if someone hasn’t sent you the status report, send them a reminder

General

• Google Code-In
  • program for 13-18 aged students
  • bounty based
  • students choose from a list of tasks, complete it, submit for approval
  • A list is being made. If you see anything that work as a project let us know (that includes things like test infrastructure, documentation, etc)

Projects:

• All seem to be going well
• Brendan + Lindsey:
  • Christian will have a review up toadyish
  • should update their diff to their newest changes
  • Concrete list of scripts: (would like to see a script for almost anything in the web ui [though not necessarily all during UCOSP])
    • close/open review request, add screenshot, get diff, show info, display request/replies/etc, upload diff, publish,
    • rb apply <review_request_id> (applying a diff (ex rb apply-diff))
  • commands should be short to code
  • holster for all of the commands
    • ex git-apply, git-clone, etc.
    • “there may be some usefulness in there you can do such as get the base path and attempt to use the right stripping, or use things like ‘git apply’ to do the patching”
  • RB bug causes some git-generated diffs to distort

• Hongbin:
  • Everything is fine
• Kevin:
  • UI project will probably be inline help followed by expanded user page
  • a timeline will be mailed out soon

• Laila:
  • How silent should the installer be?
    • some extensions (PyCrypto, PIL, etc) are kind of messy
    • easy_install generates console stuff
      • spawn in minimized window?
    • GUI installers (distutil’s, pycrypto/pil) show stuff
      • may have undocumented silent modes (some searching has been done thus far).
      • installer within an installer without a silence option
        • bug in dstutil
    • Ugliness acceptable, not ideal
    • Working > esthetics
    • sneek peek soon on ReviewBoard

• Uninstaller is beyond scope (python uninstallation is a mess)
  • cannot yet be done through distutils
• To check if something is installed:
  • python –c “import reviewbaord” gives an error code of 0, “import foobar” gives 1
    • -c executes the script passed in quotes to it
  • check path using “echo %PATH%”
    • if changed, search using it won’t find stuff
    • PYTHONPATH probably not a good alternative
    • it’s being looked into

• Lianne:
  • wanted a bit more information about her project
    • Configuration stuff was mostly just checking to see whether it still worked
      • seems not to be in not to bad of shape
    • should be able to load/save ints, strings, bools, etc. Show correctly on widgets.
      • it does
  • project suggestion: GSOC project from a couple years ago using webhooks (notifying external services of updates) never made it into reviewboard proper. Turn it into an extension
    • Code currently on github, integrated into the RB codebase
    • A webhook sends a request to some server when an even is fired
      • it serializes the update into a JSON and does an HTTP POST on 1+ urls
      • example: GitHubs post-commit notifier
      • could be used to trigger an automated build, etc.

Brendan

Status Updates – Oct 18th/24th

lindsey:
* Submitted our work for review <http://reviews.reviewboard.org/r/1847/>.
* Made touch ups to conform to pep8 code standard.
* Next Steps: Get feedback, particularly on the architecture of the project.
* Roadblocks: Same as next steps – waiting on feedback.

Lianne:
* Set up Christian’s extensions on my dev server.
* Found the SettingsForm code in the rbcia extension and in the process of implementing it on her Excel extension to test it.
* Next steps: Finish testing SettingsForm code. If it works, then start work on a new extension.
* Roadblocks: None.

Hongbin:
* Rewrote the Reviewboard bug tracker integration.
* Started to implement the proposal.
* Next steps: Continue to do the extension implementation.
* Roadblocks: None

Laila:
* Installer now installs: Python, Setup Tools, Patch, Pycrypto, PIL, and Reviewboard + dependencies.
* The database/source control components will (after a conversation with Christian) install only the python dependencies, not the actual database. That is also done but untested.
* We check whether Patch or Setup Tools are on the PATH, and whether Python is installed, before installing.
* Started investigating “Adding File Attachment to Reviewboard” which is the project she’ll be tackling after the installer.
* Next steps: Lots of testing — Having trouble right now because paramiko’s website has gone down and she can’t seem to install it properly. Also checking things like if Python is already installed and what version. Do we need to work on an Uninstaller as well?
* Roadblocks: SILENT INSTALLS: currently Pycrypto and PIL are not silent and the same goes for anthing Easy_Install.

Brendan:
* Got our code up for review and started making changes based on comments.
* Next Steps: Get feedback on the current architecture of our project. Continue to touch up code based off comments.
* Roadblocks: Waiting on feedback

Kevin:
*Invalid user/group submitted.
*Collapsible reviews posted.
*Implementing “One-Click Ship it!”
*Implementing inline help.
* Next Steps: Implementing inline help/one-click
* Roadblocks: None.