Facebook and the perpetual beta

Facebook is used by over 800 million users at all hours of the day and yet there are no scheduled outages. This social network application is demonstrating the web 2.0 pattern perpetual beta, that is, the software is continually being tested by users without any perceptible software release.

Facebook’s schedule for pushing changes out to users

Each day Facebook engineers are continously delivering application changes to their users with small daily pushes as well as more significant changes being released on the quietest day for the platform, Tuesday. They are able to release early and release often, very often: through a focussed culture, through development and operational practices and via a technology base that supports continuously delivering application changes.

The amazing technology base that supports this does so through enabling A/B split testing. This enables new or changed features to be road=tested by a small number of users (the A group) to measure the impact of site changes. Facebook engineers can choose some of us to act as a select group of users to test a change such as a new feature. They can choose how large the group is, for example to release to 1% of the public, as well as the makeup of the group, such as by age or by country, before releasing to the wider public (the B group).

What’s interesting about Facebook’s ability to release at what they call a ludicrous rate, to a massive number of users, is the use of dynamic tools. They’ve tailored their use of BitTorrent to quickly release to tens of thousands of webservers, so that they can pushout a new version of facebook in around 15 minutes – quite a radical change from the days of windows version upgrades for a single server that took a folder of dvds and outages measured in hours (if all went well).

This powerful pattern has radically changed the culture and practices of software development, these changes probably best described by the continous delivery and devops proponents. It has been adopted by large software engineering companies although it’s implementation in Enterprise 2.0 is limited by patchy adoption by enterprise application vendors. It will be interesting to see these large applications vendors providing support for features like A/B split testing and product instrumentation to the companies that use them.

This brave new world for software companies brings new challenges and issues. When it works well, customers or product managers have control over when releases are made while enjoying constant availability and developers have a close relationship with their application users. When it doesn’t work well, millions of users are impacted by an application outages as engineers release a software change that brings the whole platform down as famously happened to facebook in 2010.

Furthermore, we can see repeated examples of the backlash that facebook suffers when releases of new features change the user interface. So while the perpetual beta pattern delivers significant benefits to the platform and to the user, the challenge of delivering a service that delivers commercial outcomes and still meets the needs of a diverse userbase is not insignificant.

In spite of all these very cool changes happening to software development practices, I can’t see software developers being featured in a Project Runway type of reality show – maybe next year we’ll be cool and hip? Could Jez Humble be a fabulous Tim Gunn type of mentor telling pairs to ‘make it work’?