Meeting Minutes for June 10, 2012

New announcements:

  • Support for Windows will probably be dropped. This shouldn’t affect any of the projects.

Bogdana:

  • is still waiting on a final decision regarding the technologies proposed for the action feed backend
  • agrees with m_conley and ChipX86 that django-activity-stream is not the way to go
  • still thinks that we might be overdoing it by going for twitter or facebook-like scalability
  • was wondering about how big are the RB installations out there
  • ChipX86 points out that the biggest he knows about is somewhere around 350,000 review requests, a couple thousand users, and a 11GB+ DB. Also, RBCommons.com handles many customers, and that will hopefully only grow, so scalability is important.
  • thinks that RabbitMQ and Celery would be a big addition to the project if they’d be used just for the action feed, but is ready to give it a go since that doesn’t seem to be the case (smacleod states that it would be used for diff processing and ReviewBot as well)
  • ChipX86 thinks that the queue is less important, since we do need persistent data, and we may be back to storing as a database (this is important info and shouldn’t just suddenly disappear after a visit or a server restart). Also, caching on update should speed-up things nicely.
  • smacleod says that the main benefit from the queue is you can precompute these relationships about who is affected by the action and we can denormalize the data even so we never have to perform joins. There are plenty of ways to optimize things if we’d go down this route as well
  • ChipX86 proposes we store the history as a blob field in the ReviewRequest table. Computing the feed during view time would be cheaper than the current dashboard view
  • ChipX86 will sum up his thoughts on the matter in a more structured way and send an email
  • proposes storing the actions as separate entries in a table, but ChipX86 thinks that would mean a lot more data to store and that it would be more expensive to query
  • has to perform some stress tests for the upcoming week in order to check how expensive it would be to update and sort the history (considering it’s stored as a single field)
  • ChipX86 points out that we could improve things by using the updated time of each review request  and ETags

Steven:

  • had a long discussion with mbait about his work on the Python API and will continue based on that design
  • ChipX86 would like to see bits of that go up for review soon. And having good docstrings will help.
  • will post a design doc on the github wiki provided by ChipX86 and then start getting as much code up for review as he can
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s