Cost Overruns: How I Learned to Stop Worrying and Hate Bent Flyvbjerg
Let me preface this post by saying I have nothing against Bent Flyvbjerg or his research. My problem is purely with how it’s used in the public media, and frequently even in other academic studies, which assume overruns take place even when they do not.
Stephen Smith sent me a link to an article in The Economist complaining about cost overruns on the California HSR Central Valley segment. The article gets its numbers wrong – for one, the original cost estimate for Merced-Bakersfield was never $6.8 billion, but instead was $7.2 billion in 2006 dollars and $8 billion in YOE dollars, according to CARRD, and as a result it portrays a 25% overrun as a 100% overrun. But the interest is not the wrong numbers, but the invocation of Flyvbjerg again.
Nowhere does the article say anything about actual construction costs – it talks about overruns, but doesn’t compare base costs. It’s too bad; Flyvbjerg himself did a cost comparison for rapid transit, on the idea that the only way to reliably estimate costs ex ante is to look at similar projects’ ex post costs. His paper has some flaws – namely, the American projects he considers are older than the European projects, and there’s no systematic attempt at controlling for percentage of the line that’s underground, both resulting in underestimating the US-Europe cost difference – but the method is sound. Unfortunately, this paper is obscure, whereas his work on cost overruns is famous.
In the case of high-speed rail, it seems to me, from pure eyeballing, that there is a difference between countries in how much costs run over, and that this correlates strongly with high construction costs. German train projects, including the one example cited by the Economist, run over a lot. French and Spanish high-speed lines do not, and also cost much less.
Of course, this by itself doesn’t mean this correlation should keep holding: up until Barcelona Line 9, originally budgeted at €1.9 billion but now up to €6.5 billion, Spanish subway lines were built within budget. France has not yet had a factor-of-3 overrun on a major project, but it might in the future, and I’m not going to bet my life that it won’t. But what this does suggest is that looking at German overruns as if they’re typical rather than extremal cases is deeply misleading.
There’s an argument to be made that California’s inability to rein in the contractors will in fact lead to German cost overruns. California HSR’s projected costs look downright reasonable, whereas rapid transit projects in the state are unusually expensive. The proposed BART to San Jose tunnel is $4 billion for 8 km – very high by general subway standards, and unheard of for a subway in low-density suburbia. Going by Flyvbjerg’s own attempts to find ex ante cost estimates that are reliable, this could be used as evidence for future cost escalations; general overruns couldn’t, not without being more specific.
Pedestrian Observations 
Hi Alon,
Just to clarify a few things. The 2009 number and the current figure are not apples to apples numbers.
1) The 2009 number was in YOE dollars, about 20% higher than the 2009 $..
2) The 2009 number included the costs for the Heavy Maintenance Facility (HMF). The new numbers list costs for it between $600mm and $1 bn, but don’t include it.
3) The 2009 number included going all the way to the east of Bakersfield. The new numbers stop in downtown Bakersfield, leaving out 6+ miles of expensive track through existing urban neighborhoods.
We have tried to create an “apples to apples” by adding in $600mm for the cheapest HMF, adding in $600mm for the cost of the Bakersfield alignment and then including an extra 15% to make it YOE (less than previous number – while not everything will be done for 10 years, some of the construction is front loaded). We get numbers of $12.9 bn to $17.9 bn, as compared to previous estimate of $8 bn. This is a lot higher than we expected. They actually got rid of a lot of the aerial structures. We are quite frankly confused as to why it is so high. We will request a detailed spreadsheet to see what is driving things up.
I’m not going to comment on 2-3, but 1 is smaller than 15%, for two reasons, both of which boil down to low future inflation. First, note how the difference between the YOE and 2006 numbers, i.e. between the 2009 and 2008 business plans, was 11% in the Central Valley, vs. 27% in general; in light of that, you shouldn’t assume 15% increase over 2010. And second, because the US is currently in a liquidity trap, and the Central Valley segment is expected to be built over the next few years (midpoint 2014 or 2015), even a net 10% inflation looks pretty high. Of course, inflation could increase, but an environment with higher inflation is likely to be one with higher economic growth, which means more federal and private-sector money to complete the project.
The 15% is in for two reasons.
1) It is inline with the requirements in the Federal grant applications.
2) The 2009 numbers included an even larger factor.
If you want to take inflation out of the picture, the increase will be even more stark.
This is the spreadsheet the authority used for the 2009 numbers: http://www.calhsr.com/wp-content/uploads/2010/02/Phase_I_CapitalCost-2009Update.pdf
The pre-inflation numbers for the segment was $6.8 bn 2009$. Now the lowest possible cost estimate is $11.2 bn 2010$($10 bn + $600mm HMF + $600mm Bakersfield). This is a 65% increase on the low end.
We are really hoping / presuming there is some fluff somewhere in the numbers.
I’m sorry to sound like a nag, but how do you get $6.8 billion? The numbers in the spreadsheet say $5.8 billion YOE, excluding electrification, systems, program costs, etc. Including everything other than the trainsets, $8 billion sounds reasonable.
I’m hoping that there’s fluff in both the numbers and the design, and that the fluff will be cut. The existence of a tunnel in the northernmost Central Valley contract fills me with hope about the former and dashes all hope about the latter. We’ll soon see whether the bids for Bakersfield-to-almost-Merced come in at $6 billion as the HSRA is assuming.
page 3 landscape just adding up the 2009 cost for Merced to Bakersfield and Bakersfield to Merced. The ones on the first two pages have inflation escalations included.
Leroy Demery and Michael Setty have a good response to Flyvbjerg at http://www.publictransit.us/ptlibrary/specialreports/sr5.Flyvbjerg_Deja_Vue.htm .
Meh. It’s not a response to Flyvbjerg; it’s a response to Cox. If you want to get a better guess of what the costs will be than “Between 90 and 200 percent of the original estimate,” it doesn’t really matter whether there’s optimism bias or strategic misrepresentation. It matters if there’s an official cost estimation process that makes it illegal to get decent projections, but that only takes care of the small overruns, not the occasional Interstate network.
What’s more problematic with Flyvbjerg’s actual research is that the line in the paper that it’s hard to tell where overruns will occur but easy to tell there will be overruns makes using the work for better cost estimates difficult. The strategies used to fight scope creep, those used to fight a ballooning construction cost index, and those used to fight overruns coming from difficult change orders are different. Spain’s famous contractor oversight would be useless if HSR lines in Spain were almost 100% elevated as they are in China.
The real point isn’t that something is expensive or not.
It’s that the proponents of mega-projects lie systematically and deliberately in order to get their projects approved. (And it is lying — “strategic misrepresentation”, “optimism bias”, whatever –, for the objective and irrefutable reasons that their “estimates” only miss the mark in one direction (under-estimation) and their “estimates” never improve (in the sense of become more accurate), even in the face of extensive case histories.)
Let’s say that the University of California said “Give us $100 million from the public government budget and we’ll provide undergraduate educations to 20,000 students”. The State of California, in its infinite wisdom, balancing the costs of this proposal against other worthy competing budgetary demands, does so. But then the UC Regents turn around after a year and say: “Oh, so sorry about that. We really meant $500 million. Fork over, NOW!”
How likely is that demand to succeed? How likely is it that the next budget request from the same source will be granted? How likely is it that any promises or estimates from them will be believed?
But that’s exactly the scam that large-scale public works contractors pull time after time after time. Failure is not only tolerated, it is actively encouraged and richly rewarded.
We live in a real world of limited bug fungible economic resources. The deliberate and large-scale fraud committed by economic actors in one segment of society has real and large and negative effects upon the functioning of society (and functioning of the global ecosystem, for that matter) as a whole.
Important difference… in your University example, the University is asking for the money, and will be one directly spending it. With a public transit project, the asker (a transit agency) is not the person directly spending it – that woudl be the builder. Transit agencies do not build anything – they put it out to tender, and hope that the construction companies’ estimates match its own.
The trouble is that a transit agency won’t ever be as good as estimating the costs as a construction company. Further, the transit agency publishes its cost estimates well in advance of bidding. If the construction company arrives at a lower number, it will bid the transit agencies estimate. If it arrives at a higher number, it will bid that. Therefore, bids for transit pojects will average more than the transit agencies estimate. (Detailed design before construction tenders helps reduce this)
@Tom West, the most significant portion of cost increase comes during design and before the “builder” is even on the job. So Richard’s point stands.
It’s that the proponents of mega-projects lie systematically and deliberately in order to get their projects approved.
Richard, you are on a first-name basis with grand juries, right?
RIGHT?!?
There’s a difference between bullshitting, or giving a fact without regard to its veracity, and lying, which is having knowledge of both a fact and a provable falsity and then suppressing the fact when it becomes unfavorable. A liar must have knowledge of both the facts and the falsehoods.
Do proponents of megaprojects bullshit or do proponents of megaprojects lie in the sense of keeping two sets of books?
He isn’t just pulling that out of his ass. Strategic Misrepresentation has been studied extensively since the mid 80s, and several proven examples have been found. Go and check them out in the academic journals section of your local library if you don’t believe him.
Like Mlynarik, I too think that our experience boils down to Strategic Misrepresentation. It preys on our inability to ignore sunk costs. It is a political tactic, nothing more.
And of course not all cost overruns happen due to strategic misrepresentation. Sometimes there is an optimism bias with multiplicative or exponential effects (like when startups overestimate the size of the market and underestimate variable costs, leading to an exponential whammy). Sometimes it is just forecast error, but that is usually kept to a 20-50% range, 50% being the range bordering on outlier.
But it is a mistake to discount Strategic Misrepresentation.
Possibly the difference turns on very subtle differences that don’t lend themselves to systematic evaluation or replication? Perhaps good procurement officers instead of bad ones, rather than supposed differences in French vs. German construction costs.
Perhaps the correlation you observe is epiphenomenal. It may be easier for my hypothetical good procurement officers to operate in lower cost areas.
I think what you’re saying is a more explicit explanation of what I’m saying about construction costs. The good procurement officers lead to both lower construction costs and smaller cost overruns.
That said, this correlation between low costs and few overruns fails for rapid transit and especially light rail. The difference in construction costs between France and Germany decreases, there are large cost overruns on some projects in Spain, and conversely some American lines get done without running too much over their original, already high budget.