Roadmapping to Nowhere

(with Jennifer Gormley)

Product roadmap meetings — here’s the ideal: You and four other people cram into a small conference room that’s also stuffed with Post-it notes, whiteboards, and maybe even a slide deck. The scent of dry erase markers hangs in the air. It’s the fragrance of promise, a bouquet of hope. The idea is to come up with a plan for what the team is going to build in the next couple of quarters. A clean, prioritized list with strong consensus behind it. Ideas flow freely and fall into place like they were sent down from upon high.

Now, here’s the reality: You say, “We’ve got 37 great feature ideas to run through and plot out against our dev resources, user needs, and product vision. We need to explain to the board why we’re doing what we’re doing and stick to the plan. So I think the best way to…” and then everyone in the room interrupts you at the same time:

The Engineer: “Uhhh, yeah. We don’t really have any resources available with all these bugs the designers are flagging. Seriously, I can’t put it on a half pixel.”

The Designer: “If you’re going to tell me again we can’t do Feature 19 — that animated kitten gif for the users who get on the leaderboard — then I don’t really need to be here. And you all know I like blue, so make all the new features blue.”

The CFO: “He’s right. We don’t have any resources — since they are mostly playing
foosball in the lobby…”

The CEO: “Well, I’ve already sold Features 35, 36, 12, and eight to our new client in my last meeting with them, so those go to the top of the list…. And I have 2 more to add that their secretary’s cousin thought would be cool…. But of course we can do those after the four I just listed get finished in Q1. I can be reasonable.”

The onslaught draws a sigh out of you as you wonder if it is okay to drink at 10:17 am. As you uncap a marker and pass the sticky notes around, it’s clear that this effort failed before you all even squeezed into the room.


Product Design is fraught with best guesses, good ideas nestled among complex bad ones, and tenuous relationships with distant, sometimes theoretical users. Product roadmapping sessions are a strange mix of passion, interpersonal influence, poor planning, and misdirected energies. We’ve been in what must be hundreds of product meetings, and come up with one universal truth: Product Design = Hard. There’s no getting around that. Creating product roadmaps, however, well that can be easy. Seriously. Easy.

Recently, we faced the challenge of road mapping exactly 37 features with a new team. We didn’t want to make the same mistakes we had each made with previous teams, which almost always went like the example above. With this new team, we had just launched v1.0 of the product together. And remarkably, we all still liked each other. We even felt good about moving to the next phase; our product was in the market and feedback was coming in, mostly positive.

We looked at the blank sheet of paper upon where next year’s plan was about to unfold, and we thought: instead of tackling things the usual way, let’s take the personal passions out of it. Questions immediately clouded the way. This time, can we make — with a new, unsullied team — a less contentious, more productive, and more confident (team wide) roadmap? Could we filter out personal biases and focus only on goals (for our company, for our users, for our bottom line)? What basic math would result in a non-controversial, prioritized list to tackle with available resources? And could we use the list to make smart choices about spending more on developers, with quantifiable market impact?

Hell yeah, we could — and it actually works.

We considered 5 key perspectives:

  • Market Perception: Are we a leader? Do we have feature parity with others?
  • Cost Savings: Does the feature make things more efficient for internal ops? Solve technical debt that is becoming costly?
  • Key Vertical Differentiator: Does the feature clearly distinguish us among competitors?
  • End User UX Perception: How great is the UX for the bulk of our users?
  • Administrator UX Perception: How great is the UX for our purchase decision makers?

…and one overarching key driver:

  • Revenue Generation Potential: Obvs.

We began by asking the senior subject matter expert in relevant departments to rate each of the 37 features from the perspective of their team. These ratings were done based only on the title and brief description of the feature, because we didn’t want to spend a lot of time building specs or wireframes at this early stage. Ratings were done on a scale of 1–5 (with one being little to no potential impact, and five being high to “we better do it or else” potential impact). They did this in a vacuum. That is, no one knew how the others were rating things.

The benefit of this blind rating, done only in one’s area of focus, is that no one is influenced by the others’ numbers. Groupthink becomes negligible, and features which might otherwise overlap on a feature/value matrix begin to come into stark contrast with one another. When we reassembled to look at the features and their ratings as a group, it was clear something remarkable was happening. We’d cleared away all the extraneous debate and laid bare the priorities our collective brain deemed most critical to work on. Magic — or is it math?


Our Mathematics of Roadmapping:

For each feature we calculate a “Feature Score” (FS) as:

FS = (average of perspective scores) x (revenue generation score/5)

The first multiplier is a simple average of all the perspectives, each equally weighted. The second term overlays revenue on a normalized basis. By “normalize,” we mean we divide the revenue rating by the number of possible points so it’s value is never greater than 1. In this way, revenue can significantly boost or limit the FS but its impact doesn’t overshadow the average of the feature perspectives. Let’s take this for a spin.

Example:

  • Feature 1: “Improve User Registration” had an average score of 2.6 and a revenue score of 5, giving it a 2.6 FS
  • Feature 2: “Real-time Chat” had an average score of 1.2 and a revenue score of 1, giving it a 0.24 FS
  • Feature 3: “Offline Sync for Native Mobile Apps” had an average score of 2 and a revenue score of 3, giving it a 1.2 FS

By normalizing the revenue generation score (n/5) we can see the subtle variations in priority among all the features. Although the CTO wouldn’t have rated user registration improvements as a high-priority project, a balanced perspective shows it to be very important to take on. We have seen that natural patterns and affinities begin to emerge, like a focus on collaboration tool features, or on internal content authoring efficiencies. These roll up into easier market discussions, client pitches, and board reviews of the company vision.

This also works well when thinking about “themed” releases. If, for example, the next major market release has a “social” theme, the team can filter out features that don’t fit “social,” and then conduct the exercise. The result will be a prioritized set of projects that accomplish the theme.

Our approach has worked well for small teams, particularly so for startups. They tend to be constantly responding to changing priorities and moving targets, yet they need a roadmap to guide 3–6 months worth of development. For larger teams, it’s likely that more complex math, a different approach to averages, or laying in additional team perspectives would be more suitable. The inputs are not one-size-fits-all, but we think this technique is, and we urge you to try it. This alone isn’t a recipe for product success, but it might just save your team’s sanity, and get you a roadmap everyone believes in.

So, instead of the contentious clamor described earlier, imagine a quiet afternoon plugging in the data from your team. They provide accurate scores from their perspectives quickly and easily. You save hours avoiding the hair-pulling, passionate, but unproductive debates that devolve into eye-rolls over the designer’s animated cat gif. You come together to review the early plotting with a sense of order already on the table, making roadmapping actually easy.


When it comes to imagining and building great products, we wouldn’t trade the passion of startup teams for anything. And any tool or process that converts those strong feelings into a viable plan — or working code — is a great thing. So, in your next product roadmap session, give this math a chance. It takes the passion out of the planning and puts it where it belongs — into the work. Now stop sniffing those whiteboard markers and get to it.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s