bk99.de entertain the web since 1997

Too early, too late, too pretty: the Collingridge dilemma and why org charts are often a facade

Summary

Two short videos from the German channel Menschen und Muster, made around the Weizenbaum Institute, present classics that anyone introducing new technology into an organisation should know. In 1980 David Collingridge described the dilemma that a technology is easy to steer early on, but its consequences only become visible once it is firmly entrenched; in 1977 John Meyer and Brian Rowan showed that organisations often build their structures to meet social expectations and decouple them from the actual work. This post checks both videos against the original sources, corrects a detail about the history of the car and shows what both mean for introducing AI.

Verstehen wir Technik immer erst, wenn es zu spät ist?Menschen und Muster · 6:23Play video, loads the YouTube player
The player from youtube-nocookie.com only loads after you click; Google then receives your IP address, among other data. Without JavaScript, the click opens the video on YouTube. Privacy

The dilemma

The video above starts with the Arab Spring: back then social media were seen as a hope for democracy, today we talk about disinformation, polarisation and election interference. This leads to the question of whether we always understand new technology only when it is too late.

The answer leads to David Collingridge. He worked in the Technology Policy Unit at Aston University in Birmingham, one of the first technology assessment research units in Europe, received his doctorate there in 1979 and published the book The Social Control of Technology in 1980. It contains the dilemma later named after him: while a technology is young, it would be easy to steer, but too little is known about its consequences to justify doing so. Once the consequences are visible, steering has become expensive and slow, because everyday life, organisations and cities have long adapted to the technology.

Dust instead of road deaths? A look at the records

As an example, the video says a British commission examined the car in 1908 and above all feared the dust it raised on unpaved roads, not road deaths or exhaust. The records of the British Parliament paint a slightly different picture. The Royal Commission on Motor Cars had already reported in July 1906. Dust was indeed a main topic; a market gardener from Brentford testified that the dust made his produce worthless.

In 1908 the House of Lords debated the consequences of the car again, and dust was again at the top. The same speaker, however, cited figures from newspaper reports: 932 motor accidents in 1907 and 215 deaths. So the danger was known. For the dilemma this changes little, on the contrary: what nobody foresaw back then were not the accidents but cities built for cars, commuting and whole industries depending on them.

What Collingridge recommends

According to the video, Collingridge considered it a mistake to try only to predict consequences better, because predictions will be wrong anyway. Instead, decisions about technology should remain correctable: detect errors early, limit their consequences and keep several options open so as not to depend on a single solution. The paper by Audley Genus and Andrew Stirling (2018) lists further routes from the book, such as making technology less sensitive to error, avoiding entrenchment and distrusting large projects with high sunk costs.

The video applies this to AI: in a school, do not roll out one provider everywhere at once, but try it in individual classes and evaluate. In companies, do not rebuild work processes so that you depend on one provider. And change the guiding question: not “What will happen?” but “What do we do if everything turns out quite differently?”

The AI lab as a performance

Neu aber nicht besser: warum sich oft so wenig ändertMenschen und Muster · 8:00Video abspielen, lädt den Player von YouTube
Erst nach dem Klick lädt der Player von youtube-nocookie.com; dabei erhält Google unter anderem deine IP-Adresse. Ohne JavaScript öffnet der Klick das Video auf YouTube. Datenschutz

The second video, embedded here, starts from an experience everyone in larger organisations knows: restructuring never stops, but when something must happen quickly, you call the colleague who has been there for 15 years and sits somewhere completely different in the org chart. John Meyer, professor of sociology at Stanford since 1966, and Brian Rowan explained this in 1977 in the American Journal of Sociology. Organisations adopt structures not only because they work but because society has firm ideas of what a sensible organisation looks like. Meyer and Rowan call such ideas rationalised myths.

The video’s example is current: whoever announces going all in on AI needs an AI strategy, a Chief AI Officer and an AI lab directly under the board. This shows customers, funders and applicants that they mean it, whether or not the work improves. So that the new structure does not disturb everyday work, it is decoupled from the actual work. According to the paper, control and evaluation are replaced by a logic of confidence and good faith. The paper also lists signs of decoupling, for example goals kept so general that nobody can check whether they are achieved.

Read together

Placed side by side, the two classics contradict each other at one point, and that is where it gets interesting. Collingridge demands feedback: measure, evaluate, correct. Meyer and Rowan describe organisations that avoid evaluation precisely because it would disturb the facade. An AI lab that is mainly a performance therefore hardly delivers the experience needed to correct a decision later.

Conversely, decoupling can protect: as long as the AI strategy sits beside the actual work, that work remains reversible. It becomes risky when the performance reaches into the processes, for example through a contract that binds all data and processes to one provider. Keeping both in view means asking two things at every restructuring: what actually improves in the work? And how do we get back if it does not work out?

Ideas

  • Early on, technology is easy to steer, but its consequences are still unknown.
  • Later, the consequences are known, but the technology is firmly entrenched.
  • Better predictions do not solve the dilemma, correctable decisions do.
  • Keeping several options open prevents dependence on one solution.
  • Organisations adopt structures because they are considered sensible.
  • The official structure is decoupled from the actual work.
  • Informal channels keep organisations working despite constant restructuring.

Insights

  • Technology consequences cannot be predicted, but decisions can be built so that mistakes stay cheap.
  • Whoever creates a structure mainly for outside appearance usually forgoes the feedback that makes corrections possible in the first place.
  • Decoupling protects the work from fashions, as long as no contracts or data make the fashion irreversible.

Quotes

  • But late does not automatically have to mean too late. (Menschen und Muster, translated from German)
  • The ability to coordinate things in violation of the rules […] is highly valued. (John W. Meyer and Brian Rowan, 1977)

Facts

  • Collingridge received his doctorate in 1979 at Aston University in Birmingham; The Social Control of Technology was published in 1980 by Pinter in London.
  • Meyer and Rowan published Institutionalized Organizations: Formal Structure as Myth and Ceremony in 1977 in the American Journal of Sociology, volume 83, issue 2, pp. 340–363.
  • The British Royal Commission on Motor Cars reported in July 1906.
  • In the House of Lords debate of 29 July 1908, 932 motor accidents and 215 deaths in 1907 were cited from newspaper reports.
  • According to Meyer and Rowan, decoupled organisations replace control and evaluation with a logic of confidence and good faith.

References

Critique

  • The video dates the British commission to 1908 and contrasts dust with road deaths. According to the parliamentary records, the Royal Commission on Motor Cars had already reported in July 1906, and in 1908 the House of Lords heard about 215 deaths in 1907 alongside the dust.

Remarks

  • Collingridge’s book itself is only available second-hand or on loan; his arguments are documented here via the paper by Genus and Stirling, which cites it with page numbers. I read the paper by Meyer and Rowan in the original.
  • An earlier post here presents another video from the same channel: “Why experience protects against AI: three limits of automation”.

Recommendations

  • Introduce new tools in a small group first, measure the effect and only then decide on expanding them.
  • Before any introduction, define the way back: data export, contract term, alternative provider.
  • At every restructuring, ask what concretely should improve in the work and how you intend to measure it.
  • Make sure an AI lab delivers results that someone evaluates, otherwise it remains a performance.

Watch the video

Search the Web Archive