Mindbending ways to misconstrue project benefits
The other week I had the pleasure of attending ProjectChat in Sydney, incorporating the Benefits Realisation Summit. Last week I had the displeasure to witness what happens when an organisation has its benefits on backwards.
As a Sydneysider, you might expect me to have a dig at Melburnians every now and again but the truth is I love Australia’s other city even if I prefer to remain where I was born and raised. One thing I like about Melbourne is its vast array of public transport options that, to an outsider, looks well run. My Melbourne friends indicate this is not the case, but it took a recent article in The Age to finally convince me.
It turns out that in order to receive government bonuses, Metro Trains must meet performance targets by having a certain percentage of trains run on time. The Victorian Government launched a new system, PRS, to track the trains and measure their punctuality. So far so good—but you should see the results. According to The Age, in the first week after PRS rolled out, Metro hit its 88% punctuality target and its 98% reliability target. Yay for KPIs!
Except… “[it] did so while running hundreds of incomplete services.” That’s right, folks, the trains were actually “dumping passengers short of their destination in an effort to meet lucrative government-mandated performance targets” at a rate of hundreds of terminated services per week.
What’s wrong with this picture? The KPIs were clearly stated and met, yet you could hardly call a train that never arrives at all its intended destinations a ‘success’.
1. Benefits are realised by users
At the recent ProjectChat, Dr Phil Driver spoke on linking projects and results to uses and benefits. He was very clear that benefits can only be realised by users. You can have a whizbang product with a load of features, but unless users actually use the product, no benefits have been realised.
How would a user define the benefits of public transport? Getting from A to B without having to deal with traffic? Riding a Metro train means you won’t have to sit in traffic, but it’ll only get you partway from A to B. Fail.
2. Are you measuring the wrong thing?
Why was the government only measuring punctuality? Why wasn’t ‘reaching all intended destinations’ part of the definition of ‘reliability’? What about customer satisfaction? I doubt many would consider “better never than late” as a good measurement.
3. Are you doing the wrong project?
Prior to PRS, Metro Trains had to keep manual records on train arrivals and departures. Implementing PRS was designed to give the government automated data. The system is a good way to keep track of punctuality and reliability and, by extension, measurement provides an incentive for Metro Trains to run trains on time.
Except it doesn’t. You can motivate a train driver all you want but unless you fix entanglements in the network and find remedies for common incidents that have knock-on effects to other services, those trains will never run to all intended destinations, on time, every time. That’s another project, perhaps one that should’ve been tackled first.
Have you ever witnessed a project with its benefits on backwards?
