Bazaar developers’ blog

April 15, 2009

Update on brisbane-core

Filed under: Uncategorized — Martin Pool @ 3:30 am
Tags: , ,

[I drafted this post a couple of weeks ago, so there is some later news. It's still useful so I'll post it anyhow, with more to come later -- mbp]

The short story: brisbane-core has some good size and speed numbers; it’s now merged in to bzr 1.14rc1, so we can get wider testing across Launchpad and from users.

We hope soon to have an “edge” Launchpad code server, which would be working on live data but running release-candidate bzr code for better testing.

As fairly typical numbers for python3.0: fastimport time (similar to repeated commits) is down from 168m to 51m and the repository size is down from 160M to 77.4M. For MySQL the numbers are better, 501MB down to 170MB (nearly 3x). ‘log -v’ which was a particular point for emacs is now about 20x faster than in the 1.9 format.

We don’t have to do much new development as such but we do have to do some more risk reduction, in checking for performance or functionality problems, and some further review before it merges. This testing and tuning will continue after it lands. There are still some operations that are known to be slower than in current formats, such as branching the whole repository, but they should be fairly shallow and are being worked on. (To be clear, they’re only cases where the new format is not as good as it will be, not regressions on current formats.) It makes sense to continue working on these in parallel with letting Launchpad and other people integrate it.

The largest question mark is over how smooth we can make upgrading of stacked branches on Launchpad. This is important but probably something that we should tackle during the beta period, when other things are tied down. We will have a reasonable answer, the question is just how automatic will it be and where will it be done.

Aaron is also now working on nested trees since the sprint and that seems to be going well, with the -subtree variant now obsolete. We’re aiming to do this for 2.0 too but won’t block finishing brisbane-core on it.

March 20, 2009

Brisbane sprint report

Filed under: Uncategorized — Martin Pool @ 1:03 am
Tags: , , , ,

All of the meeting rooms in the Ibis Brisbane are named after
British/Australian explorers. It was probably a good sign that ours
was named after one of the most successful, Matthew Flinders. At least he didn’t get lost and die horribly…

We had a sprint for Canonical people interested in or working on
Bazaar: Aaron Bentley, Andrew Bennetts, Ian Clatworthy, James Westby, John Arbash Meinel, Jonathan Lange, Martin Pool, Robert Collins, and Vincent Ladeuil. I’m glad all came, particularly those who had to travel a long way.

The main point of the sprint was to develop and solidify the
brisbane-core format.
This was accomplished: we settled some open questions (especially on
using the groupcompress compressor), and showed it performing well in
many situations. Aaron is resuming work on finishing off nested tree
support, allowing separate branches to be linked in as subdirectories.

Some more random notes:

  • merged the groupcompress plugin into
    the brisbane-core branch, so now that
    branch is all you need for the new formats
  • finished a patch to kill autodetection of nested trees, which should
    mean we can use just one format that supports nested-trees and they’ll
    only be active if the user actually turns them on
  • log -v is now roughly twice the speed it was before the sprint.
  • measured brisbane-core storing the large Launchpad tree in roughly
    6x less space than 1.9
  • worked on content filtering integration with dirstate, so that we
    can handle newline conversion; finished the main patch to support it
  • talked about workflows for merge proposals for package branches, and settled how to link source branches to packages officially, and almost completed work to expose this through the api (which is the most useful bit)
  • fixed the tilde bug in urls
  • got a stacking bug fixed which was a regression from 1.12
  • bzr now does streaming push and pull, including for stacked
    repositories, and the server will no longer try to open stacked-on
    repositories itself; many fewer roundtrips; so the largest part of
    outstanding network problems are done. (To see the best performance
    you’ll need 1.14dev on both client and server.)
  • added a new conflict-diff command to bzrtools, and released bzrtools
  • fixed a bug in unicode symlink handling
  • mentored Bob Tanner to do the 1.13 release, the first in some time
    done by a non-Canonical contributor
  • settled that we’ll have the next one-week sprint in Barcelona at the
    same time and place as UDS
  • worked on a mysql-affecting bug
  • It was good.

    Going to these things always makes me wonder what Canonical would be like if we had just one big office…

The Rubric Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.