Facebook and the perpetual beta

Facebook is used by over 800 million users at all hours of the day and yet there are no scheduled outages. This social network application is demonstrating the web 2.0 pattern perpetual beta, that is, the software is continually being tested by users without any perceptible software release.

Facebook’s schedule for pushing changes out to users

Each day Facebook engineers are continously delivering application changes to their users with small daily pushes as well as more significant changes being released on the quietest day for the platform, Tuesday. They are able to release early and release often, very often: through a focussed culture, through development and operational practices and via a technology base that supports continuously delivering application changes.

The amazing technology base that supports this does so through enabling A/B split testing. This enables new or changed features to be road=tested by a small number of users (the A group) to measure the impact of site changes. Facebook engineers can choose some of us to act as a select group of users to test a change such as a new feature. They can choose how large the group is, for example to release to 1% of the public, as well as the makeup of the group, such as by age or by country, before releasing to the wider public (the B group).

What’s interesting about Facebook’s ability to release at what they call a ludicrous rate, to a massive number of users, is the use of dynamic tools. They’ve tailored their use of BitTorrent to quickly release to tens of thousands of webservers, so that they can pushout a new version of facebook in around 15 minutes – quite a radical change from the days of windows version upgrades for a single server that took a folder of dvds and outages measured in hours (if all went well).

This powerful pattern has radically changed the culture and practices of software development, these changes probably best described by the continous delivery and devops proponents. It has been adopted by large software engineering companies although it’s implementation in Enterprise 2.0 is limited by patchy adoption by enterprise application vendors. It will be interesting to see these large applications vendors providing support for features like A/B split testing and product instrumentation to the companies that use them.

This brave new world for software companies brings new challenges and issues. When it works well, customers or product managers have control over when releases are made while enjoying constant availability and developers have a close relationship with their application users. When it doesn’t work well, millions of users are impacted by an application outages as engineers release a software change that brings the whole platform down as famously happened to facebook in 2010.

Furthermore, we can see repeated examples of the backlash that facebook suffers when releases of new features change the user interface. So while the perpetual beta pattern delivers significant benefits to the platform and to the user, the challenge of delivering a service that delivers commercial outcomes and still meets the needs of a diverse userbase is not insignificant.

In spite of all these very cool changes happening to software development practices, I can’t see software developers being featured in a Project Runway type of reality show – maybe next year we’ll be cool and hip? Could Jez Humble be a fabulous Tim Gunn type of mentor telling pairs to ‘make it work’?


10 thoughts on “Facebook and the perpetual beta

  1. An application like Facebook, that has started to have such a huge impact on people lives, actually get a responsibility when it comes to uptime as well. Or what you think?

  2. Gone are the days of installing updated pieces of software being copied on to Discs. Facebook is a clear example of the changes we have seen in software being continously developed without users like me even knowing. I didn’t know that this testing existed, and after reading your blog, it makes alot of sense in developing software like with these A/B split testing. As there are advantages in developing software like this such as closer relationships being formed with developers and customers, it is also true that when something goes wrong, its a very serious problem with the issue of outages.

    • I also see it as very critical for facebook to avoid outage, but as I did research for my own blog (also wrote about Facebook, you can check out meretehole.wordpress.com) I got the understanding that Facebook has quite many routines in place to avoid outage. Its definitely one of their major priorities.

    • I know! I find it amazing they’ve found such value from their application of the bittorrent technology even though the same technology is surrounded by controversy when it’s applied to media sharing that may be against copywright laws. Perhaps facebook has learned from napster that breaking the law and going up against media companies can jeopardise an internet platform’s future?

  3. As Tony said above me, heaps of people seem to be doing facebook. I like your blog its interesting and informative. I also had absolutely no idea that facebook used bit torrent to spread its infinite number of releases. Having the ability to release a totally new version of Facebook in about 15 minutes is pretty insane when you think how many people are using it, true to the perpetual beta form though i guess 🙂

  4. Hi Amanda, lovely post! Sometimes i think Facebook is constantly changing or adding new features, i just can’t keep up. But that’s just how the world really is and has been since.

    However, perhaps with Facebook making small changes is far better than a completely new release as most softwares do. It can be a nightmare! What do you think?



    • Thanks heaps Thaddy! I guess if there’s a tradeoff between constant availability and stable features I’m happy to lean towards preferring constant availability. It does annoy me that facebook is continually adding features meaning I have to figure out how to do things all over again but at the same time it doesn’t stop me using the application. If there were regular scheduled outages I think that would stop me continuing to use the app. I’d go even further and say I’d like to see enterprise applications adopt this pattern more widely in order to provide greater availability and better stability!

  5. Hi Amanda, great blog post I found it very easy to understand.
    I found it interesting the way that Facebook deploy the updates. Facebook changes so quickly, most users find it annoying when large changes happen but that is normal to humans as we don’t like change. There is always people complaining about changes but most of them just accept it after a few weeks.

    • You are so right, but I have to admit I think Facebook is doing the transformation into timeline very well, and let most users get pretty used to it, before they are eventually enforced to implement it. This also gives the opportunity to let the technology freaks to try it out first and show how to use and/give feedback to facebook. Pretty clever!

      If you want to read more about facebook and perpetual beta, check out: http://meretehole.wordpress.com/2012/04/30/perpetual-beta/

Comments are closed.