Early Commitment
I want to go back to the problem of early commitment as I explained it two months ago. It comes out of research done by Chantal Cantarelli and Bert van Wee about Dutch cost overruns, but the theory is more generally applicable and once I heard about it I started seeing it in play elsewhere. The short version is that politically committing to a megaproject too early leads to lock in, which leads to compromised designs and higher costs. The solution, then, is to defer commitment and keep alternatives open as much as possible.
The theory of lock in
The papers to read about it are Cantarelli-Flyvbjeerg-Molin-van Wee (2010), and Cantarelli-Oglethorpe-van Wee (2021). Both make the point that when the decision to build is undertaken, it imposes psychological constraints on the planners. They are not long or difficult papers to read and I recommend people read them in full and perhaps think of examples from their own non-Dutch experience – this problem is broader than just the Netherlands.
For example, take this, from the 2010 paper:
Decision-makers show evidence of entrapment whenever they escalate their commitment to ineffective policies, products, services or strategies in order to justify previous allocations of resources to those objectives (Brockner et al, 1986). Escalating commitment and justification are therefore important indicators of lock-in. The need for justification is derived from the theories of self-justification and the theory of dissonance which describe how individuals search for confirmation of their rational behaviour (Staw, 1981; Wilson and Zhang, 1997). This need arises due to social pressures and “face-saving” mechanisms. The involvement of interest groups and organizational pushes and pulls can also introduce pressures into the decision-making process, threatening the position of the decision-makers, who may feel pressure to continue with a (failing) project in order to avoid publicly admitting what they may see as a personal failure (McElhinney, 2005). “People try to rationalize their actions or psychologically defend themselves against an apparent error in judgment” (Whyte, 1986) (“face-saving”). When the support for the decision is sustained despite contradicting information and social pressures, the argumentation for a decision is based on the need for justification.
The focus on face-saving behavior leading to escalation is not unique to the literature on transportation. In international relations, it is called audience cost and refers to the domestic backlash a political leader suffers in case they back down from a confrontation they were involved in earlier; this way, small escalations turn into bigger ones and eventually to war, or perhaps to a forever occupation.
There are a number of consequences of lock in:
- Projects will follow designs set long ago, especially ones that were hotly contentious. For example, California High-Speed Rail has stuck with the decision to build its alignments via Palmdale and Pacheco Pass, since the possibilities of changing Palmdale to the Grapevine/Tejon alignment and Pacheco to Altamont Pass both loomed large (there was a NIMBY lawsuit trying to force a change to Altamont). However, at the same time, there are plans to potentially run the partially-built system without electrification, since that issue was never in contention and is not part o the audience cost.
- There are unlikely to be formal cancellations. California is again a good example: high-speed rail lives as a hulk, not formally canceled even when the governor said of the idea to complete it, back during the Trump administration, “let’s be real,” defending the initial construction segment between Bakersfield and Fresno as valuable in itself. Formal cancellation is embarrassing; a forever construction project is less visible a failure.
- Prioritization is warped to tie into real or imagined connections with the already-decided project. California is not as clear an example of this as of the other two points, but in New York, once the real (if not yet formal) decision to go forward with Second Avenue Subway was made in the 1990s, the Regional Plan Association tied in every proposed expansion plan to that one line.
Surplus extraction
Cantarelli-van Wee treat early commitment as a problem of bad planners, who become psychologically wedded to potentially incorrect solutions. However, it is instructive to shift the locus of moral blame to surplus extraction by political actors, such a local politicians, power brokers, and NIMBYs.
In the story of HSL Zuid, much of the extra cost should be blamed on excessive tunneling. In the flat terrain of Holland and near-coastal Brabant, no tunneling should have been needed. And yet, the line is 20% underground, partly to serve Schiphol, partly to avoid taking any farmland in the Groene Hart. The Groene Hart tunneling has to be understood in context of rural NIMBYism (since at-grade solutions to habitat loss exist in France).
In this formulation, the problem with lock in is not just at the level of planners (though they share most of the blame in California). It’s at the level of small actors demanding changes for selfish reasons, knowing that the macro decision has already been made and the stat cannot easily walk away from the project if costs rise. These selfish actors can be NIMBY, but they can equally be local power brokers wanting a local amenity like a detour to serve them or a station without commercial justification. In Germany, an extra layer of NIMBYism (albeit not on connected with lock in – we have late commitment here) is demands to include freight on high-speed lines, in order to take it off legacy lines, which design forces gratuitous tunneling on high-speed lines in order to moderate the grade.
California is a good example of a non-NIMBY version of this. The state politically committed to building high-sped rail in the 2008 election, for which it showed clear maps of the trains detouring via Palmdale and going to San Francisco via Pacheco Pass. By the time further environmental design showed that the Los Angeles-Palmdale route would require tens of km more tunneling through Soledad Canyon than anticipated to avoid impact to an ecologically sensitive area, the state had already pitched Palmdale as a key high-speed commuter suburb, and Los Angeles County made housing plans accordingly. The county subsequently kept agitating for retaining Palmdale even as other alignment changes in the area were made, turning Palmdale into its pet project.
The planning literature undertheorizes and understudies problems arising from localism. In conversations with people in the European core as well as the United States, there’s an unspoken assumption that the community is good and the state is bad. If the community demands something, it must represent correction of a real negative externality, rather than antisocial behavior on behalf of self-appointed community leaders who the state can and should ignore. It doesn’t help that the part of Europe with the least community input is the Mediterranean countries, which Northern European planners look down on, believing any success there must be the result of statistical fudging.
The solution: late commitment
To reduce costs and improve projects, it’s best to delay political commitment as late as possible. This means designing uncertain projects and only making the decision to build at advanced stages of design – maybe not 100% but close enough that major revisions are not likely. The American situation in which there is no regular design budget so agencies rely on federal funding for the design of the projects they use the same federal funding for leads to bad outcomes over and over. California, which went to referendum without completing the environmental design first, takes the cake.
Late commitment is thankfully common in low- and medium-cost countries. Germany does not commit to high-speed rail lines early, and, judging by Berlin’s uncertainty over which U-Bahn extensions to even build, it doesn’t commit to subways early either. Sweden is investigating the feasibility of high-speed rail but rail planners who I talk to there make it clear that it’s not guaranteed to happen and much depends on politics and changes in economic behavior; overall, Nordic infrastructure projects are developed by the civil service beyond the concept stage and only presented for political negotiation and approval well into the process. Southern European planners com up with their own extension programs and politically commit close to the beginning of construction.
Pedestrian Observations 
Isn’t a possible risk of this that people who are actually secretly opposed to a project can simply say “more study is needed” as a means of delaying something, in the hope of a “forever never” project that people can pretend they’re being reasonable about the need for while always finding reasons not to go forward with? The two biggest Anglosphere examples that pop to mind with this are high-speed rail in Ottawa-Quebec and the North-South Rail Link in Boston, two pretty obviously useful things that the people in charge keep finding reasons not to do. I don’t disagree with the potential positives of early commitment–the fewer rocks in a river the easier the water flows–but it seems like a balance, and probably most importantly good faith, are important.
Project opponents who engage in predatory delay do so after the project is politically proposed and not before.
Boston’s NSRL may look like an exception to that timing, but it isn’t, because the opposition is not NIMBYs or other local actors engaging in predatory delay, but an elected center-right government that does not want to build it and, owing to the specifics of Massachusetts politics, does not wish to say that it does not want to build it.
Also, as the Leipzig City Tunnel can attest to even shovels hitting dirt does not preclude a century of further delays (tho to be fair two World Wars and a basically bankrupt gdr also played a role)
How does late commitment interact with long-term network master planning? Also, how does it interact with long-term land use planning?
You design the long-term master plan, and then you go to ballot on it (Swiss version) or you ask parliament to fund it (normal country version), as @df1982 explains in more detail farther down.
I can’t speak for Alon’s views, but the most functional process to my mind is that the (hopefully as disinterested and depoliticised as possible) transport bureaucracy comes up with a conceptually integrated suite of projects to develop, and the role of elected officials is to determine how quickly to fund it and which elements should be prioritised, to the extent that an integrated network plan can incorporate shuffling things around. Switzerland would be the model here, once again, but it’s probably a lot easier to do this in countries that only need incremental improvements to their networks, as opposed to areas that are trying to do HSR moonshots like the UK, California, etc.
You could possibly also use the bureaucracy to come up with different integrated plans, which the respective parties would then align with, giving voters input into the type of transport they would like to see in their region. E.g. one that prioritises a large number of new tram lines vs one that proposes a smaller amount of subway tunnels. The problem is more when individual lines become political footballs, which gives other parties the motivation to thwart them when out of office and cancel them if they get into power.
The NEC should actually fall in the incremental category. If Amtrak (was a functional organisation and) had a guaranteed budget of say $1b a year to make improvements on the NEC with a focus on increasing line speeds, you could very quickly see tangible improvements to journey times, which could continue progressively until you get down to the 3-4h Boston-Washington time that should be the end goal. Some of the projects here are big, like a new NH-Providence corridor, but others are minor works like bridge replacements, new catenary, minor curve easings, etc, which could be slotted into an ongoing improvement programme. In theory its profitability would then increase to the point that ticket revenue could pay for further improvements (or work on other corridors).
They supposedly already make money on the NEC. Partly because they’ve managed to brand Acela well enough that people with too much money like to brag that they use Acela. Which is priced to be competitive with airfares not busfare. Faster trains with more capacity should be delivered by the end of next year which will allow them to make even more money.
There are long stretches of track in Pennsylvania and Maryland that have geometry that is good enough for higher speeds. Nobody has come up with the money to replace the 80 year old electrification or upgrade the track.
Thanks for the overview of the Swiss process.
Your last part about the incrementalism of projects makes me think of the CREATE program for fixing Chicago’s rail messes, which identified about 70 small projects that could be individually funded and progressed to make improvements. I can’t say how successful it’s actually been at actually achieving its goals, though–hopefully more knowledgeable commenters could chime in.
The solution, then, is to defer commitment and keep alternatives open as much as possible.
So that there can be endless rounds of studies while nothing gets done. Pages 47 through 51 of the NIMBY/BANANA manual.
Except that until the decision to proceed happens, the NIMBYs don’t even know about the project. PAMPA, for example, sued after Prop 1A passed rather than before.
They sued after there was something to sue over.
No, the EIR was there from before, and they could have raised a ruckus and supported No and probably succeeded given how narrowly it passed.
How if it doesn’t have any funding there’s nothing to sue over. I’m sure they all assured themselves that Real Americans(tm) would soundly reject the measure and they could all merrily go back to imagining driving around in 1962.
What, in 2008? No, the polls were giving 1A a majority if I remember correctly and the state government was openly pushing for this. And yet, PAMPA was nowhere to be seen in the no campaign, because they didn’t really know or care what was going on until the political decision was made.
They live in a fantasy world. As evidenced by many of the lawsuits they filed. Which they didn’t feel the need to file earlier because they convinced themselves that the world is filled with Real Americans like them and it would be voted down.
But late could lead to never
Like the western segment of South Island Line in Hong Kong, it was planned when new estates were built over there in the 1970s, and the updated plan have it that they’ll consider actually putting money into the line when those estates are to be reconstructed in the era of 2030s.
In 1965 the New York City Transit Authority proposed building a subway station on what is now called Roosevelt Island. The tunnel was completed in 1972. They built the apartments it was going to serve but it took until 1989 for the station to open. The connection to the Queens Blvd lines, the main reason for the tunnel being built, didn’t open until 2001. The second set of tunnels, under the subway, might open to service sometime soon.
Lol you guys are stuck in the 1950s.
Battery technology is improving VERY rapidly for the purpose of powering electric cars. Tesla expects to achieve 400 W/kg very soon. When that happens 100% battery powered wheel on rail trains wont be far away. So anything on a 20 year horizon is probably not going to use overhead wire.
https://insideevs.com/news/440727/elon-musk-400-whkg-cells-not-far/
Plus hydrogen trains thay can run at 79 mph are already developed and running in “revenue service”.
https://www.alstom.com/solutions/rolling-stock/coradia-ilinttm-worlds-1st-hydrogen-powered-train
And the coming autonomy in electric cars is going to render most public transit obsolete. Certainly buses will be.
Battery technology today has DMU rather than EMU performance and costs close to twice as much as EMUs; hydrogen is even more expensive. That’s why Norway, world capital of battery-EV adoption, is wiring the commuter lines in Trondheim, which run hourly.
I get that this is reliable tech produced by established companies rather than speculation that VCs like to invest in, but not everything works the way VCs are used to. Sometimes, the class you worship is not capable of delivering solutions.
Basically battery electric trains have none of the advantages of electric over diesel that convinced 100% fossil countries to electrify and are worse on the arguments that are only recently convincing people (“no exhaust”, “fuel independence”, “less carbon emissions”)
They are of course preferable to abandoning a line or buying new diesels. But even the case for scrapping diesel trains a decade or two early to replace them with battery electric trains is weak. And the case for hydrogen (the vast majority of which is produced from fossil fuels btw) is weaker still
By the by, judging by the number of times Silicon Valley has reinvented the bus, the day of total bus obsoleteness is not yet at hand. Not nearly. Not even on the horizon.
Shorter: anything that has to lug its own power supply/storage as considerable extra onboard weight penalty is going to be at a large performance disadvantage compared to the things that tap the entirety of their power wholly from an external interface. It’s not law-of-physics possible for a BEMU or DEMU to operate more efficiently than a similarly-rated EMU. EMU’s are forever nimbler because they do not have to lug around any power supply bulk.
Maybe the performance gap narrows someday far away after considerable technological advance, but for the foreseeable future traction power batteries or alt-fuel storage are simply too frickin’ heavy to compete with external electrification.
Well the upper limit of acceleration isn’t physics but pax comfort. If a sufficiently strong motor and a sufficiently lightweight battery can be developed then acceleration could easily rise to the level where passenger comfort imposes a ceiling…
But then what would be the point of such “innovation”?
Workable battery electric trains were in use in Germany in the 1950s and they were more popular with passengers than diesel trains of similar specs.
Hydrogen trains are time and again proven to be worse value for money than battery electric in virtually all usage cases.
The need for 79 mph trains (a speed, by the way, comfortably reached by century old steam technology) doing something other than S-Bahn style service (which should be electric anyway) is pretty slim when all thick intercity lines should be hsr (and even the thin intercity lines will then have part of their overall run on hsr infrastructure).
Hydrogen, like so often, is an excuse to do nothing while proclaiming to do something. It is a bit like promising fusion power or molten salt thorium reactors (hey the latter is at least proven to be a net power producer already) while doing nothing right now. We can’t wait for the two or three decades it takes to even build an existing design of nuclear power plant. Let alone wait for technologies that don’t even exist yet at commercial scale. We need solutions roughly by the year 2000. Which incidentally is when five old people in robes decided that the planet didn’t need a future because they wanted their replacement to be nominated by the son of one of their friends…
For high-volume lines, or longer lines, overhead wire is always more cost effective than carrying very large batteries (for long distances) on every train (for high volume).
Where batteries are going to make a big difference is on low-volume branch lines, or in yards. Adding a battery to a catenary-electric trains means you only have to wire up the mainline, not every single little siding or obscure low-volume branches. Saves costs.
The mainline trains don’t go to itty bitty obscure sidings off the secondary. The great big cross country train gets broken up into a few cars for there and few cars that way and a dozen or so for the short line etc.
A formal economic argument for this could be based in the theory of option value, which says that under uncertainty, decisions are best delayed until enough uncertainty has been resolved so as to make the benefit-cost calculation clear (at list with probability). That is, there is real economic value in preserving options if relevant information is being revealed over time. But it can be hard for actors who want to “do something” to not, and wait and learn. I’m being super abstract, but maybe this is useful. There’s a classic book-level treatment in “Investment under Uncertainty” by Dixit and Pindyck.
Another “early commitment” example would be VDE8 routing via Erfurt. Now don’t get me wrong, there are certainly advantages to that routing (including the interchange point between FRA-DRS and NUE-BER it enables at a logical location with few constraints like being a dead end station) but there are also disadvantages and those hit with the disadvantages can’t help but feel they are the victims of an early 1990s bargain between Kohl and the Powers that Be in Thuringia…
Can’t see a better example of this than BART to San Jose. A ballot measure in 2000 called for a line from Fremont to Santa Clara, at a time when Caltrain looked like it was on the verge of fading way, and Santana Row was just another strip mall. Now billions of dollars are being poured into the BART hole to build the project exactly as it was conceived in 2000 no matter what developments have occurred since then. The VTA board has the ability to reconsider projects that no longer make sense — they put the Vasona light rail extension on ice. But one struggles to wonder why leaders feel like they need to follow a plan that was approved 20 years ago, and might not get built for another 10 years.
Generally true. The Tejon route is not a good example; it was never viable. Professional analyses came up with a “threading the needle” option which may possibly work, but happened to go through two rich, rabidly anti-HSR property owners; it’s not viable.
There’s a better example from CAHSR alone. The original (circa 2005?) top level alternatives analysis for California HSR said that in the big picture, the optimal route was to run San Francisco-Oakland-Sacramento-Central Valley-LA. According to the report, the operational virtue of running a single line with no branches outweighed the nominally higher distances, as did the relative ease of making straight fast lines. It required a New Transbay Tube, a route with some tunnelling from Oakland to Martinez, and a new bridge or tunnel under or over the Carquinez Strait. The route can mostly follow BART and existing expressways from Oakland to Martinez, which is well-explored geology, and I don’t know the name of the pass, but it’s not Altamont or Pacheco — it’s the one with the Caldecott Tunnel. (If this is deemed too slow via Sacramento, a shortcut could eventually be built from Concord to Stockton, again easily.)
It was rejected officially because a New Transbay Tube was thought to be too expensive. The rest of it is ludicrously cheaper to build than Pacheco *or* Altamont, and faster. Slower from San Jose to LA, but faster from SF to LA. (Question what influence Mr Diridon had on rejecting that choice after it was deemed optimal.)
Well, guess what, San Francisco is now planning to build a New Transbay Tube *anyway*. This is the sort of thing which should cause them to reconsider their route choice, given that the official reason for rejecting this route is that it would require a New Transbay Tube, and that was the only official reason given for rejecting the route. But despite the fact that their explicit premise for route choice was explicitly invalidated, this hasn’t caused them to reconsider the “Caldecott” route.
Amtrak, NJTransit and the LIRR have been discussing new cross Hudson tunnels for 40 years. One of the options actually made it ground breaking but that got canceled. It should have been open by now.
Two rich, rabidly anti-HSR property owners whose total market cap is an order of magnitude less than the cost of the Tehachapi tunnels. The US doesn’t legally empower NIMBYs very much – the ones on the Peninsula got bulldozed in court. Petty politicians like those in charge of the HSR Authority just choose to empower them because they’re the kind of manager who thinks subject matter experts should take orders from them.
The market cap of Tejon Ranch Company is 556 million dollars, and they have a lot of political pull.
Yes, that is technically one order of magnitude less than the cost of the Tehachapi tunnels, but the Tejon tunnels required would cost about the same amount as the Tehachapi tunnels, so that’s an apples to oranges comparison.
That’s not the real problem though: the real problem is that the analysis, using some standardized computer software, which attempted to find a viable route across Tejon, tried hundreds of routes and found exactly 1 which was viable.
No sane engineer would go with that. There’s no margin for change or unexpected things found in the field. There are dozens and dozens of viable Tehachapi routes. If something went wrong with the 1 viable Tejon route, it’s back to the Tehachapis and massive embarassment at the sunk costs of money wasted on the Tejon route. The fact that the Tejon route *also* requires fighting the Tejon Land Company is simply an added reason not to tilt at the windmill of trying to “thread the needle” on the one route.
And one more thing: Brightline & its predecessors have a strong (and appropriate) lobbying interest in getting California to do the Tehachapi route too, for Vegas to Central Northern California service. Although it isn’t talked about much, I am certain it is in the back of everyone’s mind. If you build the Tejon route, you end up turning around and building the Tehachapi route too, evenutally.
Altamont vs. Pacheco is a much better example of something which should have been reconsidered, but most of the rail advocate discussion on this is also exhibiting excessive early commitment bias: it should be Pacheco v. Altamont v. the pass containing *Caldecott Tunnel*, which might well turn out to be the best choice since the new Transbay Tube is being built anyway. It’s a little funny to see early commitment bias among the advocates, but it happens there too.
Hard to believe that SF-Sacramento-LA would be preferable to a Y or triangle bypassing Sacramento. Or in other words, once you have planned for Martinez-Sacramento and Sacramento-Stockton, a Martinez-Stockton cutoff would save a lot of SF-LA running time and also be very cheap to construct (maybe save money overall, as it allows for SF-Sacramento to be slower, and removes the need for Sacramento through running). I imagine that the SF-Sacramento-LA recommendation was pandering to Sacramento politicians who wanted to be at the center of things.
“Nathanael”‘s unsourced anecdotes are mostly only vaguely tethered to reality.
It’s not me. It’s the original Tier I alternatives analysis for CAHSR. In fact, maybe they were completely wrong, and were pandering to Sacramento politicians. Makes sense to me.
But you should be able to dig it up given enough effort, though things from the early 2000s can be hard to dig up now. I read it cover to cover at the time. Richard Mlynarik apparently didn’t. Apparently a lot of people commenting on CAHSR have completely forgotten about it. I think I was following the CAHSR project before Richard or Clem or Alon.
My point is, a major decision was made based on this original Tier I alternatives analysis — which as you have pointed out was shaky to start with — but this decision was made given a specific rationale. The rationale is totally gone. The decision is still “committed”. This is a perfect example of what Alon is complaining about.
Gigs of them are right here on my laptop, and metres of them dating from the early 1990s onwards are in paper files in a closet at my home, ready to burn, and I read all of them, in insane detail, because I stupidly and naively thought at one time that technical or even vaguely numerate “analysis” might matter to anybody involved at any level at any time, and there might be some hope for the planet.
The great thing about “you should be able to dig it up given enough effort” is that one can never make enough effort to unearth something that doesn’t exist. The truth is out there!