Why are so many projects set up to fail?
Project failure rates have remained steady for at least the last 30 years. Books’ Law (in The Mythical Man-Month: Essays on Software Engineering) was published in 1975, Cobb’s Paradox, “We know why projects fail; we know how to prevent their failure—so why do they still fail?” was coined in 1995, project training and certifications have increased exponentially in the last decade, but projects are still routinely set up to fail. Is self-delusion the root cause of this or are the failures caused by more deliberate deceptions?
Everyone in Australia knows it takes a couple of hours at least to travel from Melbourne to Sydney for a meeting and can extrapolate that basic knowledge to assess the travel time from London to Paris: even if you have never been to Europe you can be reasonably confident that scheduling a meeting in Paris one hour after you finish a meeting in London will mean you are going to be late for your appointment in Paris. It is just plain common sense!
Substitute any other pair of cities and make a quick search of plane and train schedules add allowances for customs, security and transfers and you come up with a reasonable approximation of the time needed and the costs involved. Specialist local knowledge can make some difference, particularly to price, but cannot change the fundamentals.
However, until this same level of basic ‘common sense’ pervades the higher levels of executive management and within the governing bodies responsible for commissioning projects and programs, these bodies will still believe you can start a highly complex major program such as the UK’s Universal Credit program and expect it to be complete in less than a third of the time any similar program has taken in the past because “this one is different”. The sad fact of life is ‘different’ usually means slower and more expensive, not quicker and cheaper, and there are dozens of surveys and reports produced each year to confirm this.
Unfortunately, acquiring the same level of ‘project management common sense’ most people have regarding travel times is generational. In the interim, this knowledge will be a very rare commodity, which allows the executives and governors of organisations to believe their own wishful thinking and/or the unrealistic promises made by advocates of a proposal and brand people who really understand the issues as ‘opponents of progress’ or pessimists.
Benefits of deception
Arguably some projects (and the host nations) benefit from this type of deception. Sydney would not have its Opera House, Japan would not have its bullet trains and potentially the UK would not have its Universal Credit if the real facts had been made available from the start.
The proponents of the schemes realised (or feared) their paradigm-changing proposals would be too much for the stakeholder community to accept and would provide their detractors with too much ammunition at ‘full price’, and deliberately start the work on an unsustainable footing, relying on Fitzgerald’s First Law of Program Management: There are only two phases to a big program: Too early to tell and too late to stop!
The problem with this strategy is there are always sacrificial lambs who carry the blame for the failure; in the case of the Sydney Opera House it was the career of architect Jørn Utzon. Despite the brilliance of his design, he never received another major commission in his life. The ethics of this strategy are highly questionable!
But despite the questionable ethics, a deliberate strategy to mislead is far preferable to the self-delusion and wishful thinking that seems to create most major project and program disasters: hope is not a strategy! And hoping the self-delusions and wishful thinking will somehow turn into success is a major governance failure that should be sheeted home to the ‘governing body’ that allows it to occur, rather than blaming the sacrificial lambs lower down the hierarchy.
Failure hasn’t changed
The basic reasons for failure have not changed in the 30 years since the Harvard Business School published one of the first studies on the subject. The four basic reasons identified in the Harvard report and many subsequent lists are:
- Unclear objectives leading to inadequate scope definition and the consequential underestimation of time cost and risk.
- Wrong leadership from the sponsor down, leading to poor decision making, inadequate skills in the team and a failure to invest in team formation.
- Poor or no planning leading to unrealistic timescales and ineffective controls.
- Poor communication and a lack of stakeholder consultation/management.
For as long as I can remember the reported failure rate for major programs and projects has been stuck at the 70% level. This is despite the huge increase in qualifications, the adoption of the Gateway Review Schemes and the establishment of PMOs. There is also an abundance of publicly available case study materials documenting contemporary government fiascoes in the UK, the USA and Australia (at least) prepared by their respective National Audit Offices. So why is this knowledge not being used to properly scope, schedule and cost most projects and programs?
Unfortunately for project/program professionals, well over 75% of the causes for failure have nothing to do with how well you run the project or program. I did some light research on this a couple of years back (see my conclusions).
The root cause of these failures is poor governance, most probably caused by the governing body being largely ignorant of the project management practices developed in the last 60 years, which in turn makes self-delusion much easier to sustain. As Richard Bacon reminds us in his book, Conundrum: Why every government gets things wrong—and what we can do about it, “culture eats strategy for breakfast”. While it is culturally acceptable to be ignorant of project and program management, governance failures will persist.
The problem is more than just knowing. In the absence of rigorous self-knowledge, self-delusion is normal! In his paper ‘The Evolutionary Basis of Self-Deception’, James Sage argues that from an evolutionary, Darwinian point of view, self-deception may be indicative of a normally functioning, evolved human mind. A little self-deception goes a long way, particularly if you successfully sell your concepts and get your next great project underway—the alternative may be unemployment!
However, this tendency to use self-delusion to ignore implementation issues in favour of grand ideas is probably more than a simple lack of rigour and knowledge; it may well be a rational response to the fact that our public and organisational political systems confer more rewards for the shrewd deployment of symbols and generalised arguments than for detailed, realistic analysis and forecasting. The problem with this approach is that eventually the real world of implementation catches up with the grand ideas and, as Niccolo Machiavelli advised in The Prince, “a man who neglects what is actually done for what should be done learns the way to self-destruction rather than self-preservation.”
Highlighting failure
Things may be changing; academia is starting to take a far greater interest in major project failures. Bent Flyvbjerg in particular has produced a stream of papers highlighting the causes of major project failures. Some of his titles are:
- ‘Double Whammy: How ICT Projects are Fooled by Randomness and Screwed by Political Intent’
- ‘What You Should Know About Megaprojects and Why: An Overview’
- ‘Should we build more large dams? The actual costs of hydropower megaproject development (generally more than double the estimate)’
- ‘Mega Delusional: The Curse of the Megaproject’
In summary, we know how to deliver normal-sized projects but most management hierarchies prefer failure over investment in success. We know megaprojects are always high risk ventures but politicians love them, provided they can find someone else to blame if things go wrong. And we know unrealistic time and cost estimates will not be achieved and that attempting to achieve the impossible will cost far more in the long run than working to sensible targets. So why are management structures (public and private) allowed to propagate these avoidable failures so often?
In my view, the root cause of most project and program failures is the failure of governing bodies to both hold themselves accountable and to make their management responsible for commissioning avoidable failures. The art is to balance risk and reward within an effective governance structure.
