If, like me, you’ve enjoyed Carlo Rovelli’s new book ‘The Order of Time’ you’ll know that our perception of time is an illusion and past, present and future are both interchangeable and fundamentally unknowable. However, if I was to use that excuse, at Programme Board, to explain why the programme had just hit an iceberg I suspect I’d get short shrift followed by an invitation to vacate the premises. And yet, as a PMO Manager, people look to me to predict the future or – more precisely – to present an analysis of the information available to me that helps senior stakeholders make a decision that will improve the programme’s future state. I find this challenge endlessly fascinating and, given I’m regularly asked about it, thought I’d share our thoughts on how a good PMO goes about polishing its crystal ball.
If we look at the specific requirement, I would phrase it as:
“Analysing accurate information about a programme’s past state to allow the presentation of future scenarios to aid decision making.” This breaks the problem into three manageable chunks:
- Analysing – doing something to the data that adds value to it.
- Accurate information about a programme’s past state – creating the information flow process, tools and assurance activity to capture that accurate view of the past state.
- To allow the presentation of future scenarios to aid decision making – creating a dashboard or reporting format that easily supports a rapid understanding of the possible scenarios and the decisions that need to be taken through either the formal or informal governance.
Holding a mirror up to the programme
All information is, by definition, information about a past state and most of the control machinery of a programme is geared around pulling that information from the programme, through various tools and processes, and assuring its quality and consistency. We refer to this activity as ‘The Engine Room’ in our P2Consulting PMOs and, particularly in the early stages of maturity, it is where a large amount of the effort is expended. There are no silver bullets to reduce this need, and indeed whole manuals can be written on how to do this. I tend to stick to a few top tips:
- For each information source understand what, where, how often and, most importantly, why – ask yourself why this information is needed and for what purpose.
- A clear information map showing the flow of information and the processes, tools and templates that will be used to gather it are critical both for building the information flow and for the training and stakeholder engagement that will be needed.
- A key role of the PMO is assurance of that information to ensure it is complete, compliant, consistent and gives 100% coverage which all adds up to the information being correct.
- Keep track of the quality and consistency of reporting from the programme – it helps identify problem areas and show improvement in reporting maturity over time.
Lies, damned lies and statistics – analysing the data
Well, perhaps it’s not that bad but truly analytic predictive reporting does require the use of statistical analysis – which is a useful tool, but a tool nonetheless. How it is used or abused is critical. Once you’ve got your clean and accurate view of the past state, then proper analysis can be conducted. I group the type of analysis I am conducting into three types:
- Heuristic analysis – which is short-hand for using true subject matter experts (SMEs) to review the data and identifying areas of concern. In practice, this is why the PMO and project manager with years of experience is so valuable – their brains are trained to look for patterns in the data (without even being aware of it) that points to trouble. Never underestimate the power of giving properly refined data to experts and letting them draw conclusions from it.
- Content based quantitative analysis – what most people think of as analytics. Usually this involves taking large amounts of data and applying a technique such as Quantitative Risk Analysis, Reference Class Forecasting or Decision Forest to present future scenarios and their probability of occurring. These different methodologies have strengths and weaknesses based on the quality of the initial data, but ultimately, I’m looking for them to answer three questions:
- What is the risk to the current target for the future
- How big is that risk in scale and probability, and
- The levers/decisions the programme needs to decrease the likelihood of that risk
- Content-free Behavioural Analysis – potentially the most interesting of the three (especially on extremely large programmes or portfolios where it’s impossible to be a subject matter expert on all areas). This technique looks at the previous predictions of various areas of the change programme/portfolio and ‘scores’ how well they’ve made predictions (i.e. how far out where the resource estimates, how late in the day did the milestone turn ‘red’ etc.). This score is then used to put confidence around future estimates such as resource need or RAG confidence. This can be twinned with a measure of reporting hygiene (i.e. how complete is an areas reporting/ how often is it late?) to provide some really interesting behavioural insight into the health of an area – both pointing out areas for focus/concern and supporting decisions around the size of contingency to allocate to these areas.
Remember, no matter how clever the technique it is only useful if the data it is based on can hold the weight of the analysis, and if it can be presented in a way that supports decision making.
Presenting the analysis – the 3Ts
And there are numerous ways of presenting the data – but many are overly complex and take from the conversation rather than adding to it. At P2Consulting, we have a rule of thumb for presenting data called ‘the 3Ts’. Essentially all reports should cover 3 things:
- Track – tracking information is looking at historic information that says where we have been. For instance milestones completed, resource utilised or cost spent.
- Target – target information identifies what the current view of success is for the programme. A specific milestone on a specific date, a specific number of points delivered to a specific cost parameter, a specific customer satisfaction score by a specific programme stage etc.
- Trend – trending information shows the current best estimate of what is likely to happen, against a specific information stream, if the programme continues to current expectations and assumptions.
Critically, therefore, all representations of information relating to the programme should be able to provide the position of all three of these information groups in order to inform decision making within the programme and help frame ‘the question’. A worked example using an Agile ‘burn-up’ chart is a helpful illustration:
The ‘track’ shows what has happened; the ‘target’ shows the current success criteria; the ‘trend’ shows the current best prediction. No matter the complexity of the analysis, one should be able to present the relevant metric using this approach, which focuses all decision making around two questions:
- How better to align the ‘trend’ with the ‘target’?
- Is the ‘target’ in the right place?
Ultimately all risks can be seen in light of their impact on the ‘trend’ line – i.e. if this risk materialises, the trend line will be shifted down by X amount. If we don’t mitigate issue Y, then the trend will come in 100 points under the target. These are the options to pull back on target.
And all decisions about ‘target’ reflect the wider programme’s stakeholder community saying what ‘good’ looks like for the change.
The Prediction Craft
Obviously the above only scratches the surface of the many approaches to supporting your programme with predictive reporting and there are as many answers as there are programmes with specific and unique needs. The most critical thing to remember, of course, is that prediction is, by definition, an uncertain act. It needs elements of both art/experience and science/statistics – it is, therefore, a craft in the very truest sense and like all crafts, constant practice and monitoring one’s own performance to ensure improvement over time is the key.
About P2 Consulting:
If you’d like to talk more about predictive reporting or strengthening your PMO please visit www.p2consulting.com or get in touch with Adam or any of the P2 Consulting team here on Linkedin. You can also follow P2 Consulting on Linkedin
P2 Consulting is a market leading project & programme management consultancy. We work in partnership with our clients to turn their business ambitions into reality, bringing a unique blend of leading-edge thinking and hands-on delivery. We bring the drive, the passion and the courage to act leaving your business stronger, fitter and more profitable for having worked with us.
P2 Consulting was formed by a revolutionary team of entrepreneurial leaders and award winning consultants. Since our creation, our passion for delivering business success for our clients has shaped our vision, our company and every member of our P2 Consulting team. We are proud to have delivered some of the largest and most successful programmes in corporate history.