I have a pretty concrete institutional theory for why the United States, and to some extent the rest of the Anglosphere, lags in infrastructure. It mostly fits the available evidence, but “mostly” and “available” are the operative words, and I don’t want to expound on it too much before doing more interviews to contrast American infrastructure planning with Continental European and democratic Asian examples, to see if there’s basis to what I’m saying.

But one piece of the theory is worth talking about early: the concept of managerialism. The relevance to infrastructure is roughly the following set of propositions that constitute this theory as applied to public policy:

• Big outfits should be run by professional managers, who should be trained primarily in management and not in a specific industry; it is acceptable and even desirable for a CEO to bounce between different industries. A successful founder or manager in one field should be presumed capable of quickly acquiring expertise in another field if they move to a new industry.
• Domain knowledge is suspect, because the people who hold it are self-interested – in public policy this relates to public choice theory. At best, domain knowledge means you get to work for a manager.
• Managers should set up the right incentives to force underlings with domain knowledge to innovate, and do not need to acquire detailed domain knowledge themselves. For example, they should set up objective metrics to evaluate employees by rather than have close enough relationships with the employees to know intuitively who to promote.
• The recruitment pipeline for the managers should combine a set of institutions producing a single elite (Oxbridge, Ivy League) with a proof-of-pudding system measuring success by earned wealth.

The upshot is that if you don’t trust any of your workers (public choice theory, again) and do trust the managerial elite to be able to run all industries equally, then you can just do whatever you want and blame the inevitable failure on the workers being too stupid or incompetent.

Note that even though this is often an anti-government theory of how to run public-sector agencies, it is as written politically neutral, and even used by leaders on the left. Politicians of all stripes appoint people with the wrong skillset to run public agencies, preferring political appointees (who in both the US and UK come from the same institutions as the private-sector managerial elite) to career professionals. Career professionals may be too politically independent and have long-term plans that are not compatible with self-aggrandizing schemes to build visible infrastructure that a politician can claim full credit for.

Note also that even though the full set of propositions I associate with managerialism comes from the English-speaking world, segments of it can be found elsewhere. France, for example, has a Grande Ecole-educated elite that views itself as omnicompetent. It differs from the Anglo-American model somewhat in that the institution that produces engineering executives (Polytéchnique) are not the same as the one that produces politicians (ENA), and a a lot in that bouncing between industries is narrower, so that SNCF is run by airline executives without experience in railways rather than by industrialists and financiers without experience in transportation.

I make no claim about whether managerialism works in other spheres, like general business. That said, in the fastest-growing high-end segment of the American economy, tech, the business culture is very different: everyone, including management, is expected to know how to code; managers are recruited from among experienced programmers; the culture regards external managers much less than it does coder-founders like Larry Page, Sergey Brin, or Mark Zuckerberg, to the point that most people in tech and tech media regard Microsoft’s stagnation in the 2000s as the fault of the transition from founder Bill Gates to the more managerial Steve Ballmer.

But in the public sector, at least in infrastructure, managerialism has not succeeded. Any of the following reasons may be relevant to the failure of turnaround experts, political appointees, private-sector CEOs, and other non-industry professionals to improve American public transportation.

• American business culture assumes that the same methods work regardless of scale. Public transit is scale-dependent, which fries a lot of common private-sector assumptions. Most importantly, starting small is not always possible, especially in trains. Managers who are used to starting small end up deemphasizing the most productive parts of public transportation, like rail operations, in favor of things that can be done incrementally, like bus lanes.
• American culture is generally closed to foreign knowledge. It is also pragmatic and anti-theoretical, viewing foreign knowledge as a kind of theory that must be tested at very small scale before being applied widely; one American big-city transit manager denigrated international cost comparisons as “Paris or something.” The difference between managers and industry professionals is that some of the latter understand that public transportation works better in Europe and East Asia and try to learn, whereas managers see nothing to learn in countries with living standards that are (on average) comparable to the US’s or (for senior managers) much lower.
• Public transportation has a lot of moving parts that have to be planned together – timetable, infrastructure, equipment, and more broadly also development. Even within operations, there are different departments that affect one another closely, like dispatching and actual operations. This makes typical responses to bad news, like a hiring freeze, atrocious, because an overstaffed agency may have one understaffed department creating too much work for everyone else; only an experienced transportation professional would know to fix the problem department by hiring more people even in a bad economy in order to increase productivity elsewhere.
• Infrastructure has very long time horizons. Agency heads have to think on the scale of decades, not quarterly earnings calls with the shareholders.
• Competition is destructive. The real competition is cars, and not other modes of public transportation. Competitive private businesses generally understand coordination (“synergy” was a much-mocked buzzword in the 1990s and 2000s), but less deeply than researchers with familiarity with the situation of multimodal public transportation.

What this means is that the penchant of so many American politicians to hire outsiders to the field is not part of the solution to the problem of failing transit agencies, but rather part of the problem. Success comes from hiring people who are experienced in the field, and if the agency bureaucracy seems too inflexible, then hiring from other countries. There’s a reason Andy Byford, a career transportation planner with experience in London and Toronto, was such a hit success in New York – and there’s a reason this success involved developing much greater levels of mutual trust between management and the workers. In contrast, a string of people whose background is in a culture that treats everything as an American business to be turned around with tough management does not produce good results – rather, such leaders create problems that justify their own continued existence, blaming their own failures on the people below them.

Practically any change has some beneficiaries (why else would it happen?), but a separate question, often with a separate answer is, who does it empower? For examples, expansion of education benefits students but empowers teachers and school administrators, and expansion of health care benefits patients but empowers medical care providers, especially ones below the prestige level of doctors. In both cases, the groups that were empowered back expansion, and at least in the case of teachers, the other side assumes that there are no beneficiaries, hence for example Humphrey’s comment in Yes, Minister that comprehensive education was adopted only because of the teachers’ unions.

The relevance of this distinction is that improvements in public transport mostly have the same set of beneficiaries – current and potential transit riders – but empower different groups of people depending on what the proposed improvement is. This matters, because it’s easier for change to happen if it empowers people who are already important. This is not restricted to politics – a change in how a business is run is easier to accept if it empowers senior management than if it empowers grunt workers, even if the ultimate beneficiary, that is the shareholders, is the same. The upshot is that there are changes in how to run public transportation that empower the already-powerful, and those changes are not necessarily best for the riders.

Politicians and civil servants

Politicians, especially high-level ones, are more powerful than civil servants. It is therefore easier to pass changes that empower them than ones that empower the civil service.

I think the Anglosphere’s fascination with design-build comes from this, at least partially. Traditional design-bid-build procurement means that an in-house team reviews bids and selects separate contractors for design and construction. In contrast, design-build removes power from the civil service. The consultants who get to draw design specs get empowered, but I don’t think this is why governments adopt design-build. Rather, design-build means that high-level politicians get to make big decisions, first since they often have casual ties to the consultants through a revolving door, and second since each bid is bigger (one firm might do everything) and therefore it is evaluated at a higher level.

Consulting in business is a good analogy, since there are some analogies between how the state is run and how big businesses are. The relevant one is the tyranny of the org chart; see some examples here and here by Aaron Renn, and here and here by other consultants. Senior management in the private sector has serious problems with listening to people who the org chart asserts are subordinates, from middle management all the way down. Management consultants often succeed by talking to lower-level workers, getting good ideas from them, and then packaging them to senior management in glossy presentations that look like they came from the consultants, who have nebulous job titles so as to convince senior management that the consultants are their peers. In effect, consultants are a workaround to the fact that senior management is unlikely to adopt ideas that empower subordinates.

The greatest irony here is that the sort of political operatives who are most educated in public choice theory are the ones who most consistently act according to its precepts. They dislike public-sector unions, so they institute public-sector hiring freezes and instead outsource work to consultants. In effect, they empower themselves, as senior political operatives who get to make more important decisions when the decisions are about higher-level things (that is, who gets the work among design-build bidders).

National vs. international comparisons

The issue of outside comparisons depends heavily on what the agency is to be compared to. The difference is that in the United States, managers are well-traveled domestically but not internationally, so domestic comparisons empower them and international ones do the opposite. In Europe, managers are more internationally traveled but largely within Europe, so comparisons to Asia are as problematic for them as comparisons to non-English-speaking European countries are to American managers.

What this means is, a study delivered to Boston or Los Angeles or Chicago that does a domestic comparison will bring up things that top managers and politicians are at least somewhat familiar with. A manager in Boston may have happened to work only in New England, but this is not common, and the manager’s social circle will include people with experience from other parts of the United States. This manager can read a report employing domestic comparison and will have heard about some of the success cases in the report, and if anything is unclear, the manager can call up friends and former coworkers and get clarifications. The manager is thus empowered to implement the report’s recommendations.

An international comparison has the opposite effect. The American manager might be facing a report that brings up case studies from European and East Asian countries, where few Americans have ever lived. The report might mention things that all American transit managers have convinced themselves are impossible, because those managers only ever talk to other Americans. It devalues most of the expertise of the American insider. If anyone within the agency is empowered, it is often a junior planner who has delved into foreign cases out of interest, or perhaps an immigrant whose knowledge is foreign and not just American. It completely upends the hierarchy: the senior manager has no way to contribute to the process and is at the mercy of outsiders and subordinates.

The trick that management consultants use to persuade senior managers to accept recommendations that came from below is not useful here. The report cannot hide its foreign provenance; it screams right there, “your experience as a senior American manager is not as valuable as you think it is.”

Is there a way out?

I believe that there is. I don’t know this for a fact, but I have some circumstantial evidence pointing in a more optimistic direction.

First, not only is the idea of the tyranny of the org chart well-known to consultants, but also a brief Googling revealed a number of different consultants openly pitch their skills to management in how to avoid the problem. That people who get paid to give outside expert advice to corporate leaders believe they can tell to those leaders’ faces, “you need to listen to your subordinates better and here’s how you can do it” suggests that it is possible to get at least some managers to listen.

Second, on an abstract level, managing others is a valuable skill on top of the deep experience managers must possess in the industry they lead, and moreover, general management skills are highly valued in American business culture in the private as well as public sector. This means that even though the use of foreign advice devalues senior managers’ industry-internal skills, and maybe even some precepts that they’ve learned in other industries if they’ve jumped around from industry to industry, it still does not devalue general management skills, not should it.

Third, beneficiaries matter, rather than just the people whose skills are valued more under the changes required to improve American public transportation. This means that a politician who is seen as successfully improving infrastructure will get accolades from the public, because there’s general political consensus that infrastructure is good, and specific political consensus in the parts of the United States with the most public transit ridership that public transportation infrastructure is good. Political advisors may be sidelined by change that relies on knowledge they don’t have, but elected politicians who are seen building infrastructure cheaply become more popular.

And fourth, the situation in the United States in general and New York is particular is so bad that change is possible even while respecting at least some degree of turf. Gradual replacement is possible, if New York implements one change that reduces costs by a factor of 2 while leaving other causes of high costs unchanged, and then the people who successfully shepherded the change implement more such changes. Future changes can devalue the skills of managers who only know how to build and run bad transit and not good transit, but a manager who was responsible to a large cost reduction will get enough internal and public clout that empowering this manager further through further-reaching reforms will be easier for the hierarchy to swallow.