Ranking criteria for stories – part 1

One of the most tricky decisions in product management is to decide what to build and each seasoned product manager has his or hers idea of what works best. Some use mathematical formulas and feature scores, other dev estimates while others use their gut and industry experience. So which one is best and which ones you shouldn’t use? I will cover in this series of articles the most common criteria through which product managers score stories and ideas as well as several methodologies used to rank them.

The first variable that helps score each story is cost and based on how this is estimated (I covered estimation techniques here) you can use man days, t-shirt sizes or story points to calculate it. This is always just a reference number and not an absolute source of truth (remember – this are still estimates at the end of the day). Depending on the feature on top of the development costs there might also be other physical costs associated which the product manager has to always consider. On top of direct costs we always need to also look at the opportunity cost.

The second criteria is time to release and here my advice is to never compare just development times but full end to end timelines because although some feature might require less dev time than others and could seem like a quick win, in reality there might be dependencies (internal or external) that could push back the release date.

A third criteria that I use is reach or the number of customers impacted by the feature. Will your next feature be built for 10% of your customer base, 50% or will it address all of them? On top of this one should also consider the potential of the feature – market penetration (what percentage of the market is impacted by this feature?). This values are estimates and in order for this to be as accurate as possible you have to be realistic when working out the number. When you look at the total market / customer base you also need to take into account the commercial strategy of the company because although a feature might seem it impacts a large market segment (80%) in reality if one of the organisational goals is to focus just on the enterprise segment (the remaining 20%) you are not building for the right customer. Having this discrepancy would mean the score has to be scaled down and this leads to using one of the variables I created to help called target market validation which is 1 if the feature is built for the right market segment or 0,8 if it is not. If the feature covers both (fully or partially) its score will be 1. One interesting point I would like to make is that the Product Manager will always have the final say and can override any score.

The fourth criteria I use is impact, which measure how much this impacts the users and how critical it is on their day to day work. For example a feature that decreases the time it takes to load a screen from 6 seconds to 3.5 seconds might have a high impact on your user base from a UX perspective, however one that will enable customers to sell faster will have an even higher impact even if it’s used less often. The impact is measured on a predefined scale:

• 0,25 minimal impact
• 0,5 low impact
• 0,75 medium impact
• 1 high impact
• 3 critical impact

The fifth criteria used is confidence and this is a score that reflects how much we trust the other scores used. When assigning a value to this the product manager has to take into account his team’s experience, any unknowns that might have an impact, what dependencies are on other teams / features (internal and external). The values I use are:

• 1 for high confidence
• 0,8 for medium confidence
• 0,6 for reduced confidence
• 0,5 for low confidence

This first five criteria are the most common used and they provide an effective and easy way to compare features and to rank them. However although this five combined will give us an initial view that can help us create a roadmap there is one adjustment I recommend doing in order to make the score even more accurate and this is to add to the mix my second variable: company strategy alignment score. This shows how much does the feature align with the company goals and objectives. The values for this variables are 1 if the feature is in line with company strategy or 0,8 if it is not. For example is the business decides to focus on increasing profit and the feature we are building will open a new revenue stream is should receive a score of 1.

‍