Business 2.0, NGOs & Nonprofits

Prototyping vs. piloting

This is a cross-posting of a post originally published on the IDX Backstage Blog.

In conversations talking about iterative approaches to projects I often make the distinction between a “pilot” and a “prototype”.

I can’t recall where I heard this, but I remember someone once saying that a “prototype” is as much about working out what doesn’t work (failing informatively, to borrow Clay Shirky’s phrase) as it is about working out what does work.

A prototype should be “light”—the minimum investment necessary to test something. It should be, conceptually, something that you’re not afraid to throw away.

The term “pilot”, on the other hand, infers something where there are a number of knowns and you’re really testing what it takes to actually run something—to take it to scale. There is a high expectation of the thing actually working. There may be more significant investment, just not to full scale. Often this may be the trigger for a summative evaluation (i.e. a stop-go decision)—making the stakes higher.

I’ve had occasion to revisit this thinking in the past few days. Both as we consider the first early prototypes of a series of workshop/event activities, and also in support of one of our Innovation Lab participants.

In the latter case, our participant is trying to understand how best to prototype an iPad app that will be used in a workshop context. There are a lot of mechanics to the workshop, and an underlying program logic (or theory of change) that needs to be tested, in addition to the app itself. So we’ve been exploring how paper prototyping tools might service the testing of this wider area of concern, before jumping too far into the development of the app.

At the same time, the IDX team has been exploring how coding and robotics workshops with primary school aged children could be of value in achieving our mission objectives. We’re looking into whether building on existing tools and approaches (MIT Scratch and FIRST LEGO League, for example) might work in our context. How we might need to adapt them. What level of interest might exist around these particular activities.

As a team we’ve been back-and-forthing about what “prototype” means in this scenario. Even running an initial workshop requires a degree of investment in hardware (computers, educational LEGO kits) that isn’t insignificant. What is the minimum investment needed, so that we can reduce the feeling that it must succeed (i.e. maintain space for informative failures).

I mapped this out on the back of a napkin this morning:

A rough mapping of prototyping through to delivery

We end up with three broad objectives for each phase: LEARN, PRACTICE, and EVOLVE. Of course, prototyping (and learning) can occur across the process, but as a broad mapping I found this useful to get an understanding in my head.

What do you think? Is this a useful distinction? Are there other definitions of these phases that provide a better understanding of when they apply?

Design

The value of small prototypes

I was recently chatting with my friend Lopa about a variety of things social innovation-y, and the topic turned to prototyping.

One of the things that came up is that it can be easy to think that prototypes need to be a lot more than they actually need to be. That is, it can be just as valuable to prototype small pieces of the puzzle as you go, rather than thinking that you have to prototype an entire service, or complex parts of a web application etc.

As I was thinking about this on the train, I caught up on some episodes of Kevin Rose’s excellent Foundation series. In her interview, the founder of Smitten Ice Cream Robyn Sue Fisher talks about prototyping a number of times in the interview (especially in relation to her core product). But what interested me most was how she used a food cart to get her service up and running (at about 28 mins and 45 secs in):

I’ve benefited from this principle as I work through the Seasonal Saturday concept. It seems like quite a simple thing “a seasonal meal once a month”. While I’ve got a lot of thoughts about what we could do with the initiative, I felt it was important just to start doing it, as I knew things would pop up. So we bootstrapped a simple blog and invited a bunch of people to submit their recipes (one per month).

This prototype is a long way from what we’re aiming for, but prototyping early has already helped a lot in working out the logistics and some of the barriers and challenges participants might face. Like, what information do we need to include with a recipe? Are there specific attributes to recipes that we need to consider? What happens if I want to participate, but can’t do that recipe (i.e. I don’t have a slow cooker)? These may seem trivial, but as we are hoping to encourage/enable behaviour change, understanding (and addressing) these barriers where possible is important.

As a tutor in the Design Research Training unit at UWS I also saw the power of prototyping first hand. Students often came up with grand ideas and some couldn’t see, at first, how they could prototype it because they were jumping ahead to their bigger vision. Others were able to break their project down into smaller parts which they then prototyped. These students tended to do better overall with their projects, and all of them learnt a lot from this process.

It’s important, of course, to recognise the limitations and changed context of a smaller prototype. But following the lean/agile approach of prototyping early and often is a great way to help ensure that a project has the greatest chance of success.