Up to now we've been talking about Agile development. So what we do are development iterations and what we get is incremental delivery of functionality. These iterations allow the business to be agile, to be responsive to changes (and there are always changes). The process is simplicity itself - inspect and adapt.
My question is does "inspect and adapt" lend itself to creating innovative solutions or does it merely optimise them?
By innovation, I mean an idea which is a step change - a discontinuity - from what we already know or do. Innovation is about thinking out-of-the-box, changing the rules and revolutionising. Optimisation, on the other hand, is about taking what we already have and evolving it.
From my experience, project pressures usually mean the Agile process is an optimising machine. Clients want to keep going forward even when they know full well the solution isn't right. What happens is a solution which may be fundamentally flawed is polished and band-aided into something passable. It takes a brave soul to say: "This isn't working. Let's go back to the drawing board!"
I'm not saying that there is no place for innovation in Agile. The problem is we don't seem to have time for it. No one quite knows who's responsible for it. And people are by nature quite precious about their work, so any radical departure from what has already been done is always regarded suspiciously and as a backward step in the project.
You're probably thinking the answer is really simple: just put the (innovative) features on the backlog and prioritise them. That is indeed what we should do but, by its nature, innovations are often so full of unknown quantities that no one wants to or can estimate and scope them, which means they languish on a product backlog, never making it on to a Sprint backlog. A wacky, way-out idea is often difficult to visualise - the clients can't do it nor can the developers - so how do we quantify it?
It could potentially go into a Sprint as a research task - a spike solution. I think that depends on the knowledge gap. Another approach is to take it out of the Sprint cycle altogether. I don't suggest doing this for every backlog item. I see it as an R&D stream or a proving ground for the difficult-to-visualise features. In most projects, a product backlog will mainly consist of features we have seen before and so by experience we can confidently quantify them. At best we will have recourse to design patterns that pretty much detail the solution for that feature. And often we can comfortably extrapolate or create composite solutions from parts we are familiar with.
I have an ulterior motive for suggesting this "R&D stream" that runs in parallel to the iterative development cycle. It gives user-centred design its own space. We can storyboard, prototype and carry out in-depth research to test ideas. These activities are time-boxed as required and the outputs (if appropriate) eventually feed into the backlog as properly qualified and considered items.
In one way, the "R&D stream" is helping the development team estimate by clarifying the unknowns. But more importantly, the "R&D stream" is helping the client innovate by generating, testing and visualising ideas in a user-centred way.
In conclusion, I think we can indeed innovate in an Agile environment and one of the best ways to do that is by taking a user-centred approach. However, in order to not disrupt the Agile development cycle and to not distract the User Experience delivery team members, I see an "R&D stream" running alongside with a mandate to let their imaginations run wild!