There’s something skewed about value distribution
How often are you reviewing what’s valuable in your IT portfolio? We find that it is often distributed very unevenly, yet prioritizing opportunities by value is critical to managing large-scale IT.
While there’s a general understanding among product and portfolio managers that most value will come from a small number of investments, there is still a widespread practice of spending precious resources on opportunities yielding little or even no value at all.
In our experience, this gross underestimation of how unevenly value is distributed in an IT portfolio stems from a mix of traditional project and project management office (PMO) practices. Failure to adapt product and portfolio governance to the reality and magnitude of the asymmetry in value distribution leads to companies missing out on opportunities worth millions of dollars every week. But this doesn’t have to be the case. We’ll share how IT can improve their value distribution based on what we have seen in our work across different industries, geographies, and technologies to help visualize and explain what this practically means in terms of things to do and not do in IT portfolio management.
Key characteristics of product development
Product development is a broad term for work that is not repetitive or standardized. Traditionally, this work has been managed and governed as projects, where work is plan-driven, predictable, and managed as a ‘once-and-for-all’ before being handed over to operations. Product development has a set of key characteristics that affect how value is created and accounted for differently than standardized, repetitive, and operational work. 1
Product development is knowledge work
Product development is creative and knowledge-intensive work, more than capital-intensive work. Key constraints and success factors are people and their time and skills, which become more determining the more digital products and services become.
Capacity is constrained
A common claim is ‘budgets are always limited’. But for exciting and promising opportunities in large companies, financing is rarely the main challenge. Typically, the bigger challenge is to find the right set of skilled resources and provide them with the means to pursue the opportunity.
Predictability is poor
Business cases are estimates built on assumptions. For a business case to become a fact, two things need to come together: a successful undertaking and a subsequent benefits realization. And even then, the actual return is very unlikely to match the estimate. In some cases, benefits and business cases are created more to fit a PMO process framework than project a future reality.
The non-repetitive nature of creative work makes the prediction of effort to complete the work difficult. And it makes it even more challenging to predict the outcome, i.e., how much value the work will generate.
Time is limited
For product development, time is not only a factor driving costs, but it also plays a crucial role on the benefit side of the business case.
No business value is generated until the output of the work is in business operations. In most cases, this means that it is in the hands of customers and users, so time-to-market is a critical factor for starting the payback period. The window of opportunity is sometimes set by time itself, due to seasonality, or by events outside your control, e.g., by actions of competitors and regulators.
Economic decisions with Cost of Delay
With time and resource capacity as key constraints, distinguishing different types of value in terms of ‘soft’ vs. ‘hard’ benefits does not aid the decision to allocate resources and to pursue opportunities.
As an aside, so called ‘soft’ benefits such as brand reputation can have monetary impacts. Fines and other consequences of not following regulations will also have an economic impact on your business. Even though return on investment (ROI) cannot always be measured in increased revenue, it is still an economic trade-off decision to make the investment in product development.
We advocate the technique of Cost of Delay for estimating value and prioritizing opportunities. It takes time into account and greatly facilitates trade-off decisions between different types of opportunities. For example, the value of regulatory compliance (cost avoidance) can be compared with opportunities for cost reduction and revenue generation.
Skewed value distributions
Data sources
The five datasets presented below come from our consulting work for retail, logistics, and pharma clients. They represent programs of work for central IT solutions across finance, supply chain, and core operations. The solutions were all supported by a mix of insourced and outsourced IT teams providing new feature development, enhancements, and maintenance of core IT solutions. In all instances, more than 150 people were involved.
An example of an opportunity prioritized for delivery:
RQ-6008 “Adjustments needed for adding operator [X] onto [Central Reservation System] platform”. The benefits estimated were based on the projected additional volume the operator was contracted to bring once operational. Cost of Delay ca $ 440,000 per week.
More than 80/20
We expected the value distribution to roughly follow the ‘law of the vital few’, also known as the 80/20 Rule. It states that, for many events, roughly 80 percent of the effects come from 20 percent of the causes.
Our datasets indicate that value is more unevenly distributed than the 80/20 rule suggests. The 20% of most valuable opportunities (causes) in our data represent on average 88% of the total value (effect). 2
The proportion of total value from the 20% most valuable opportunities
| Logistics portfolio | Pharma portfolio | Core reservation system | SAP finance system | ERP solution |
| 81% | 83% | 91% | 93% | 94% |
But what surprised us even more than 88/20 average was how value was distributed along the whole spectrum.
High probability of insignificant value
When plotting out the estimated values in decreasing order from the most valuable in terms of Cost of Delay to the least valuable, it is easy to see that the plot does not represent a linear distribution. A handful of the opportunities were estimated to be much, much more valuable compared to all the others.
Nearly all opportunities are found in the long tail of the relatively insignificant value; 67 out of the 70 opportunities are found in the bottom 20% of value. 3
A common argument against prioritizing by value is that it’s difficult, requires effort and is ‘only estimates anyway’. And while there’s truth to all those claims, it does not account for the probabilities.
By deliberately not selecting an opportunity estimated to yield high value, there’s a 96% chance that it will be of insignificant value compared to any of the high-value opportunities.
A recurring pattern
Since we worked with the ‘Core Reservation System’, we have observed incredibly similar patterns for other core solutions and portfolios. As the graphs below show, there’s something inherently skewed in all of them.
It’s worth noting these points we’ve observed in addition to the above:
- Similarly skewed distributions where value has been estimated in monetary terms with other techniques, e.g., with Internal Rate of Return.
- The same pattern is evident in different types of opportunities (from strategic program initiatives down to epic-level requirements).
- Very low correlation between estimated benefit values and actual effort. Where we have reliable data for implementation effort, the correlation coefficient between estimated value and actual cost if delivered opportunities was +0.27. 4
A sharp elbow
There are several ways to describe how uneven value is distributed. For the mathematically inclined, we can express the skewness using an exponential equation of the format: y= ae^(-bx)
Applying exponential regression, we get values for the factor (a) between 1.42 and 1.59 and for the exponent (b) from -0.33 to -0.46. Thus, as an approximation, we can assume normalized value distributions for core IT portfolio opportunities to be modelled with this formula: 5
y= 1.5e^(-0.4x)
This model fits to our real data with R squared values from 0.894 to 0.987, indicating a very good fit. 6
Data scientists recognize this as an ‘elbow’ and it implies that the values around the bend are the most significant in the distribution. While it is encouraging to see how the model fits values from different clients and contexts, it’s not easy to relate to this insight and what it implies.
More unfair than income in South Africa
The exponential nature of the value distribution is not entirely surprising. In economics, income distributions are known to be skewed according to exponential functions. Economists describe inequalities of value distribution with the Lorenzo curve and the Gini-coefficient. 7 The curve shows income inequality is distributed across the population.
Our datasets have a Gini-coefficient of 0.84 across them. For comparison, this is more extreme than the income distribution of any country. The income inequality table in the world is ‘topped’ by South Africa with a Gini-coefficient of 0.67. 8
Based on our data and insights, it’s safe to assume that value is exponentially distributed for product development opportunities in IT portfolios. Taking this insight into account, what does it mean practically for management practices?
Implications for management practices
Estimated value in a business case is a projection and does not equate to value realized. But value projections are how we base decisions. And expect that when we do decide to pursue an opportunity, the result we get might not be the one we forecasted; it might be different depending on when the opportunity was realized.
With the realization of value being dramatically unevenly distributed among opportunities, the logical conclusion for management practices is that a one-size-fits-all scheme will not be effective.
Implications for portfolio management
For portfolio management and prioritization decisions, we recommend that different policies should apply depending on where on the elbow curve the opportunity value is expected to be. For the rare opportunities of extreme value, always pursue them. Even if it is at the cost of extra effort and disrupting other priorities.
For the opportunities below the elbow, never let them get in the way of more valuable ones.
Deliberately prioritize and carefully manage the opportunities around the elbow value. We recommend:
- Managing funding so it covers the capability and resources to realize opportunities over time, rather than allocate funding and budget to individual opportunities.
- Focusing on timing effects and prioritize opportunities with shorter time to market. While there are several uncertainties in all estimates, the benefits are zero until value realization starts.
Equally important, we advise against:
- Planning for full, or 100%, utilization of resources that cannot easily be reallocated. This can cause massive delay costs when new high-value opportunities appear.
- Relying on benefit/cost ratios. They are ineffective because they do not account for the magnitude and variability of the numerator (benefits).
Implications for prioritization techniques
The uneven value distribution, in combination with the effects of time, means the application of prioritization and scheduling techniques has a crucial effect on benefit realization.
| THINGS TO DO | THINGS NOT TO DO |
| Treat all different types of work the same way (regulatory, risk avoidance, etc.) as they all take time and capacity. | Bundle opportunities together that have discreet value. |
| For similarly valued opportunities, give preference to the work that takes the least time to complete: it will deliver value earlier and free up resources for new high-value opportunities earlier. | Use monetary values for benefits as the basis for the selection and sequencing of opportunities. |
| For similarly valued opportunities, give preference to the work that takes the least time to complete: it will deliver value earlier and free up resources for new high value opportunities earlier. | Trust categorization techniques such as (‘MoSCoW’) or arbitrary numbers as substitutes for monetary value. 9 |
| Use opportunity costs for ‘soft’ benefits instead of categorization or policy. | Give up on estimating with probabilities and monetary value because it’s hard to get started. |
How sharp is your elbow?
We should expect value to be extremely unevenly distributed. Not just in theory, but our data shows similar uneven distributions of value in a variety of large-scale IT portfolios across different industries, technologies, and regions.
We’ve found that it’s not 80/20, but closer to 90/20. Without estimating and prioritizing work, it will be like playing the lottery but with very few, very lucky winners. Most of the effort and valuable time will have been spent on opportunities paying out insignificant amounts or nothing at all.
How confident are you that you’re exploiting your most valuable opportunities in your portfolio?
If you want to know more about Cost of Delay or how we focus on delivering value early and often to maximize your probability of winning, please comment, get in touch, or download our thought paper on projects to product transformation.
Notes:
- The most in-depth explanation of these characteristics is provided in Don Reinertsen’s Principles of Product Development Flow, which also covers the opportunity costing model. It’s also well described as Lean Software Development by Mary & Tom Poppendieck.
- Calculated as the percent of the total value the most valuable 1/5 of opportunities represent. 88 percentage points is the average of the five portfolio values.
- The values for benefits have been “min-max” normalized to between 1 (the highest value) and 0 (the lowest value). The probability for a randomly selected opportunity to be in the lowest 20% of value in the four sets ranges from 0.72 (Logistics) and 0.94 (Pharma).
- The correlation coefficient ranges from -1 (strong negative correlation) to +1 (strong positive correlation). Values around 0 indicate no correlation at all. Reliable effort data is only available for the subset of opportunities pursued. We used actual cost data from the outsourced supplier in the Logistics Portfolio data set.
- Applying the curve_fit function from Pythons scipy.optimize library on the datasets with min-max normalized values. The exponent is negative as we have arranged the data as a long tail with the lowest values to the right. When applied to non-normalized values, the factor (a) will vary, but the exponent will remain the same 0.4x.
- R squared (R2) values range between 0.0 and 1.0. The closer the score is to one, the better the model fits. See sklearn.metrics.r2_score
- The Gini-coefficient, or Gini index describes how far from even an income distribution is and is calculated by the proportion of the areas on each side of a Lorenzo curve and the line of equality. The Gini coefficients in our individual datasets range from 0.74 in the Logistics Portfolio to 0.88 for the ERP Solution.
- Wealth would be even more unevenly distributed than income, but income distribution is a better comparison to IT portfolio opportunities. Data from World Bank 2022.
- SAFe® recommends using ‘relative’ Cost of Delay values and when applying the Weighted Shortest Job First (WSJF) scheduling heuristic. This would not account for the actual distribution of value.
Additional insights
Realizing the value of ideas
Moving from projects to products
Product Management
