There’s a study by Eno looking at urban rail construction costs, comparing the US to Europe. When it came out last month I was asked to post about it, and after some Patreon polling in which other posts ranked ahead, here it goes. In short: the study has some interesting analysis of the American cost premium, but suffers from some shortcomings, particularly with the comprehensiveness of the non-American data. Moreover, while most of the analysis in the body of the study is solid, the executive summary-level analysis is incorrect. Streetsblog got a quote from Eno saying there is no US premium, and on a panel at Tri-State a week ago T4A’s Beth Osborne cited the same study to say that the US isn’t so bad by European standards, which is false, and does not follow from the analysis. The reality is that the American cost premium is real and large – larger than Eno thinks, and in particular much larger than the senior managers at Eno who have been feeding these false quotes to the press think.
What’s the study?
Like our research group at Marron, Eno is comparing American urban rail construction costs per kilometer with other projects around the world. Three key differences are notable:
- Eno looks at light rail and not just rapid transit. We have included a smattering of projects that are called light rail but are predominantly rapid transit, such as Stadtbahns, the Green Line Extension in Boston, and surface portions of some regional rail lines (e.g. in Turkey), but the vast majority of our database is full rapid transit, mostly underground and not elevated. This means that Eno has a mostly complete database for American urban rail, which is by construction length mostly light rail and not subways, whereas we have gaps in the United States.
- Eno only compares the United States with other Western countries, on the grounds that they are the most similar. There is a fair amount of Canada in their database, one Australian line, and a lot of Europe, but no high-income Asia at all. Nor do they look at developing countries, or even upper-middle-income ones like Turkey.
- Eno’s database in Europe is incomplete. In particular, it looks by country, including lines in Britain, Spain, Italy, Germany, Austria, the Netherlands, and France, but even there it has coverage gaps, and there is no Switzerland, little Scandinavia (in particular, no ongoing Stockholm subway expansion), and no Eastern Europe.
The analysis is similar to ours, i.e. they look at average costs per km controlling for how much of the line is underground. They include one additional unit of analysis that we don’t, which is station spacing; ex ante one expects closer station spacing to correlate with higher costs, since stations are a significant chunk of the cost and this is especially notable for very expensive projects.
The main finding in the Eno study is that the US has a significant cost premium over Europe and Canada. The key here is figure 5 on takeaway 4. All costs are in millions of PPP dollars per kilometer.
|Tunnel proportion||Median US cost||Median non-US cost|
However, the study lowballs the US premium in two distinct ways: poor regression use, and upward bias of non-US data.
Regression and costs
The quotes saying the US has no cost premium over Europe come from takeaways 2 and 3. Those are regression analyses comparing cost per km to the tunnel proportion (takeaway 3) or at-grade proportion (takeaway 2). There are two separate regression lines for each of the two takeaways, one looking at US projects and one at non-US ones. In both cases, the American regression line is well over the European-and-Canadian line for tunneled projects but the lines intersect roughly when the line goes to 0% underground. This leads to the conclusion that the US has no premium over Europe for light rail projects. Moreover, because the US has outliers in New York, the study concludes that there is no US premium outside New York. Unfortunately, these conclusions are both false.
The reason the regression lines intersect is that regression is a linear technique. The best fit line for the US construction cost per km relative to tunnel proportion has a y-intercept that is similar to the best fit line for Europe. However, visual inspection of the scattergram in takeaway 3 shows that at 0% underground, most US projects are somewhat more expensive than most European projects; this is confirmed in takeaway 4. All this means that the US has an unusually large premium for tunneled projects, driven by the fact that the highest-cost part of the US, New York, builds fully-underground subways and not els or light rail. If instead of Second Avenue Subway and the 7 extension New York had built high-cost els, for example the plans for a PATH extension to Newark Airport, then a regression line would show a large US premium for elevated projects but not so much for tunnels.
I tag this post “good/interesting studies” and not just “shoddy studies” because the inclusion of takeaway 4 makes this clear: there is a US premium for light rail, it’s just smaller than for subways, and then regression analysis can falsely make this premium disappear. This is an error, but an interesting one, and I urge people who use statistics and data science to study the difference between takeaways 2 and 3 and takeaway 4 carefully, to avoid making the same error in their own work.
Eno has a link to its dataset, from which one can see which projects are included. It’s notable that Eno is comprehensive within the United States, but not in Europe. Unfortunately, this introduces a bias into the data, because it’s easier to find information about expensive projects than about cheap ones. Big projects are covered in the media, especially if there are cost overruns to report. There is also a big-city premium because it’s more complicated to build line 14 of a metro system than to build line 1, and this likewise biases incomplete data because it’s easier to find what goes on in Paris than to find what goes on in a sleepy provincial town like Besançon. Yonah Freemark thankfully has good coverage of France and includes low-cost Besançon, but Eno does not – its French light rail database is heavy on Paris and has big gaps in the provinces. French Wikipedia in fact has a list, and all of the listed systems, which are provincial, have lower costs than Paris.
There is also no coverage of German tramways; we don’t have such coverage either, since there are many small projects and they’re in small cities like Bielefeld, but my understanding is that they are not very expensive. Traditionally German rail advocates held the cost of a tramway to be €10 million/km, which is clearly too low for the 2010s, but it should lower the median cost compared to the Paris-heavy, Britain-heavy Eno database.