Don’t trust agile alone to build successful products

Discover product/market fit

The Validation Patterns card deck is a collection of 60 lean product experiments that will help you de-risk product success.

We are all doing high-frequency waterfall. Agile won’t save us

More often than not, I see software companies being stuck in what I call Design, Build, Ship. They are so focused on making their ideas come to life, that they forget to test whether the ideas were any good in the first place.

Product builders often neglect to test whether their customers actually have the problem they are trying to solve and whether there are enough customers to make up a market.

There are several things to test before any development efforts make sense. First; are we trying to solve a real problem? Second, if users actually agree on the problem being worthwhile solving: are there enough of them to make up a market for your product? Next, if the problem actually exists for a decent amount of people; does your solution actually solve the problem? And finally, are people actually willing to reach into their wallets to pay for your product?

There are so many unknowns that need to be checked off, before it even starts to make sense to initiate any development work.

Agile to the rescue

We tell ourselves that agile development will come to the rescue. That if we part our development work into short sprints, we give ourselves the option of changing paths along the way. Afterall, doing agile should make us agile, right?

Agile was originally about reducing uncertainty and maximizing learning, but it seems that the original true version of agile didn’t survive its growth and popularity. Agile today often ends up being high-frequency waterfall.

The original promise of agile make sense. One of the key values of agile; responding to change over following a plan¹, is exactly what we are looking for.

However, Scrum, XP, and other agile implementations fail to answer what changes we should respond to. Agile fails to ask “why”. All Scrum ceremonies – planning, refinement, daily stand-ups, review, and retrospectives – all focus on answering when, how, and what to build. There is no ceremony in Scrum that asks “why?” Why are we building this in the first place?

Scrum and agile aren’t bad. They just are not delivering on being agile in every respect. Scrum is great for letting teams focus on executing incremental change. But it often struggles to accommodate for learning and taking action along the way.

Why is that?

You cannot eliminate uncertainty by building a backlog

At the start of a development project, everyone involved in bringing an idea to life puts their effort into building up the backlog in an attempt to grasp the entirety of what they are trying to accomplish.

As humans, we naturally try to avoid ambiguity and uncertainty and have a bias toward seeking closure and completeness². Not knowing better, many teams try to avoid uncertainty and succumb to this bias, by hurrying to establish a full backlog of what needs to get done.

As works is broken down into sprints, it starts to resemble an actual plan. The problem with plans is that the intention is to implement them. It is just too easy to see the backlog as a massive pile of “work to be done”. When you see it as work to be done, you tend to focus on what and when to build. The result is product tunnel vision.

Too much focus on what and when to build without asking why, creates tunnel vision.

To escape the tunnel vision, we need to find a way of regularly asking “why”? Why are we building this?

Too often, teams look at the massive pile of work to be done and start imagining solutions in their head. They visualize in their head how to implement the pile of work, telling themselves “I don’t think that will be too hard.” However what happens too often, as teams gets their hands dirty and start building, is that reality turns out to be more complicated. Suddenly a problem comes out of nowhere, leaving teams asking “why didn’t we catch this earlier?”. They get stuck. During implementation.

You need to do more than thinking to build successful products

Teams get stuck in implementation when the planned solution, only visualized in their heads, is only imaginary. Thinking about how you are going to execute on something is not the same as doing actual work to figure out what the best solution is.

The hill chart as illustrated by Ryan Singer. There is significant work to be done figuring out what to build and how to build it before you start implementing it. This fact is often neglected by product builders and agile practitioners.

Ryan Singer illustrates this dilemma perfectly in what he calls the hill chart³. The hill chart implies that we need to do hard uphill work to set ourselves up for frictionless downhill execution. It implies that we need to do actual work to figure out the best way to implement an idea before we start doing to work to implement it.

When we only think and about how were are going to implement something without testing it first, we position the work at the top of the hill instead of at the bottom of the hill. Then we skip doing work to figure out how to best implement it. When that happens, we tend to get stuck during implementation.

The solution is to start at the foot of the hill. If all you have done is to think about the solution, then the position of your work is at the bottom left of the hill, as you have no evidence that your thoughts will work.

No matter what happens in a project, we are bound to get stuck at some point in time. However, getting stuck during implementation becomes a problem, when a full development team is set up to do the implementation. Getting stuck is a problem when we are in execution mode.

A much better time to get stuck, is when you are stilling figuring things out. When we are still figuring things out, we still haven’t assembled the whole team around implementation, but rather just a few persons around finding the right solution. When we are not in execution mode, but rather a search mode, failure is expected.

In execution mode, our main job is planning why failure is not an option. here, our main focus is improving the existing process to make it as efficient as possible. When Scrum teams focus on increasing their development velocity, they are focusing on executing more efficiently.

In search mode, our main job is experimentation. Here, failure is embraced and expected. Here, our main focus is improving the rate with which we learn: our learning velocity. Experimentation means doing actual work to get real evidence that our intended solution will have a good chance of being successful.

In search, the main job is experimentation

In search mode, we are generally looking to get evidence that our solution will work in three key areas through experimentation:

That it is desirable. That our target segment has the problem we’re trying to solve and that the solution we are developing is a good fit to solve that problem. We test value propositions, wording, and comprehension of our target customers as well as the means with which we communicate it.

Examples of common desirability experiments include:
- Comprehension tests. Test whether your target audience understand the message you are trying to get across.
- Fake door testing. Build a cowboy village, pretending to provide a finished product to test whether the finished product could be of interest to your target audience.
- Single-feature products. Reduce a product to only solve one isolated problem rather than building an entire platform, to test interest in the core interaction you are offering.
- Video Demo. Simulate a working product through a hand-held video demo to gauge customer interest in a finished solution.
- Wizard of Oz. Use human power to fake complex automations before implementing anything to test whether your intended solution will have its intended effect.
That it is viable. That it will have the desired effect on our business. We test our business model through conversion rates, pricing points, and try to get indications on the actual business value the solution will deliver.

Examples of common desirability experiments include:
- Dry Wallet. Simulate a pay-now experience to test willingness to pay.
- Collect pre-orders. Collect pre-orders before producing anything to test whether your target audience are willing to pull up actual money from their pocket to use your intended solution.
- Sell the future. Sell your unbuilt feature to a customer before starting any implementation work to test customer demand and willingness to pay.
- High hurdle. Make a user experience more difficult to use to gauge especially keen interest.
That it is feasible. That we can build it and make it happen. We spike out the most important parts of the problem to be confident that what is left is just a matter of execution. There will always be surprises in implementation, but you will be less likely to run into nasty surprises by testing for feasibility first.

Examples of common desirability experiments include:
- Development spikes
- Working prototypes. Build just enough of a feature to test its intended behavior and whether it is possible to build.
- Single-feature products. Isolate critical features that might be hard to build into a solution on its own to get evidence that it is feasible to build.
- Move in with the customer. Work alongside your customer for days or weeks to get a sense of whether the solution you intend to build will be feasible for your customer as well as you.

The Validation Patterns card deck is a collection of 60 lean product experiments that will help you get evidence in a matter of hours, not days or weeks.

Start experimenting today

Learn more

Acknowledging discovery work early

We need to acknowledge that there is significant work to be done before we start implementing. That there is important work to be done searching for the right solution, eliminating uncertainty, before we start executing our plan. That we need to figure out what the right thing is to build before we start making it happen.

Unfortunately, too many teams start as if there was no uncertainty to be eliminated. As if they know everything it takes to make their future product or feature an instant success. The result is failed product launches or teams getting stuck during implementation as they run into unexpected problems.

As we try to materialize an idea, the one thing we can be certain of, is that we are going to get stuck along the way. The question then is: when would you rather get stuck? When you have set up a 10 man large development team for execution, or with a one- or two-man experimentation team?

If you neglect search and discovery work, you will be lucky to hit a home run. Why take the chance?

Search out the things that can go wrong and produce evidence that your approach will cover them all. When you are confident that what you intend to develop is desired by your customers, is viable for your business, and is feasible to implement, you team can trust that the work is going to ship and have the effect planned.

Sources

¹ Kent Beck, et. al., 2001, Agile Manifesto

² Uncertainty Avoidance at Wikipedia

³ Ryan Singer, 2018, Real Work vs imaginary work

Do you want to learn more about product experimentation?