DrinkingWorldCup Post-Mortem

June 30, 2010 by tom Caffeinehit

In what will be the first in a new series of blog posts, Tom provides an in-depth analysis of a recent project and highlights what went right, what went wrong, and what we learnt.

Our latest project at Caffeinehit HQ was the DrinkingWorldCup. The idea, which arrived during a lovely sunny lunch in mid-May, was to provide real-time drinking challenges to people watching the FIFA 2010 World Cup in South Africa. As I write this, the tournament is only in its third week and there are still many more games to be played before the conclusion on July 11th. Unfortunately, the DrinkingWorldCup hasn't had quite the response we had hoped for so we thought it would be good to get our thoughts down while they were still fresh.

Premise

With only a month until the World Cup kicked off, we knew we had to keep things simple. We agreed straight away to develop both a website and an iPhone app; the website would be suited to playing at home and the iPhone app for playing at the pub. After that, it was fairly simple to come up with the idea of setting real-time drinking challenges for people (for example, if there was a foul then the person should take a drink).

An incomplete action

With the basic premise settled, we brainstormed some more ideas before deciding on our initial features:

• People choose a team to play for
• Every time that team makes a mistake or does something note-worthy during the game, everyone playing for that team will be assigned a certain number of drinks to consume
• Players can choose to complete the action or skip it by selecting the respective button
• At the end of the match, a report would be generated from the data we collected

What went right?

As with all Caffeinehit Lab projects, we wanted to use DrinkingWorldCup to experiment. If experimentation was used to determine the success of projects then I would say we succeeded. While using Django to develop the site wasn't very novel (it is our web framework of choice), we did choose to use MongoDB instead of a more traditional RDBMS like MySQL, PostgreSQL or SQLite.

By using MongoDB, we were able to play around with the database layout quickly and easily since schema changes were cheap to do. In a highly time-constrained project like this, having flexibility in the database schema was very useful as we were able to iterate quickly. MongoDB database files are also portable (a property shared with SQLite) which made deployment simpler. The only real hiccup occurred when we realised the 32-bit version was limited to a 2Gb database.

To get Django and MongoDB working together we used MongoEngine. All in all, it was a breeze to use but you'll need to make some very simple changes to your Django settings.py.

Because we were building an iPhone app, and because the website made heavy use of AJAX, we needed to develop an API. A good API must be useful and the best way to do this is by initially implementing a minimal set of features and then adding new endpoints upon request. It's easy to assume you know what data people want and how it will be used but this is rarely the case (see Facebook). We discussed the basic API and it was only extended when we had a specific use case for more data.

Alen Mujezinovic: 15:30:41
    Endpoints:
        /fixtures/
        /actions/<game_id>/(?last=<action_id>)

Tom Drummond: 15:44:36
    We need a rules endpoint in the api (to display in the app)

Alen Mujezinovic: 15:46:33
    Endpoints:
        /fixtures/
        /actions/<game_id>/(?last=<action_id>)
        /rules/

In a way, the lack of time was beneficial - for development at least. We launched without a huge amount of testing and a number of features not implemented which, as a developer, is a painful process. But it meant we could see what worked and what didn't work with real people. Taking our cue from other alcohol based websites, we created a welcome screen asking visitors to confirm their age and login through Facebook. Based on the logs, we figured that this wasn't helping us or our visitors. They didn't know what the site was about before logging in and we were getting fewer people through the gates, so to speak. Needless to say, we quickly rejigged the flow of the website so people were able to browse before signing up.

This iterative development allowed us to focus on the features that actually mattered to our users rather than what we thought mattered.

So, what happened?

After some discussion we agreed there were two main reasons why DrinkingWorldCup wasn't as successful as we'd hoped: the subject matter & poor PR. During development and testing, we had a lot of fun playing the game amongst ourselves and our friends yet we found it difficult to convey this to people browsing the website. Despite having hundreds up people logging in, only a small handful ever participated in matches.

In hindsight we realised that it's actually quite a challenging problem to create something that compliments the user's primary focus of attention. Using the website or app was probably too much of a distraction when watching the actual games; paying attention to a secondary screen - especially if you had friends around - was too much of an inconvenience. We also noticed that even when people joined a game by selecting a team, they often would not complete any actions (by selecting either the "Done" or "Skip" buttons).

Our other problem was getting people to know about it. Due to a lack of time we didn't invest any effort into PR until it was too late. We also believed it would benefit from word-of-mouth but that didn't really happen. When we did eventually attempt a PR push (several days after the World Cup began) we were informed that it would be difficult to get any major press because of the subject matter. This, unfortunately, was something we had foreseen.

What we learnt?

Apart from the obvious (e.g. new development skills), we learnt about user behaviour and how not to design a product that won't be the user's primary focus. We realise now that if your app is only going to be devoted a small fraction of the user's attention then you must design accordingly. Do not expect your user to take an active interest by asking them to complete actions. Asking users to push buttons was necessary for us to generate interesting match report pages but was it really necessary from the user's perspective? Probably not.

We also learnt that we need to pay more attention to PR. We are not PR people and doing all that kind of thing distracts from what we love doing (building awesome products). But it is necessary. Being the kind of people we are, it has always felt wrong to promote our stuff before it's ready but that doesn't seem to be the best way to do PR work. To help overcome our PR deficiency we have created a set of processes when building new products. It will almost certainly require tweaking but we believe it will be positive step. We're working on a new project at the moment which we'll test it on and after that, we'll post our results and detail what the process is.