18 October 2026MuM · AutomationCurated

Why experience protects against AI: three limits of automation

Summary

Sascha Friesike, professor and director at the Weizenbaum Institute for the Networked Society, explains on the rbb channel Menschen und Muster why AI will not replace human work in the foreseeable future but will change it. The video draws on three research findings: Polanyi's paradox of tacit knowledge, Lisanne Bainbridge's ironies of automation and the observation that expectations grow with technical possibilities. This post summarises the arguments, checks the cited studies and shows what follows for working with AI in everyday professional life.

Warum Erfahrung der beste Schutz gegen KI istMenschen und Muster · 9:47Play video, loads the YouTube player

The player from youtube-nocookie.com only loads after you click; Google then receives your IP address, among other data. Without JavaScript, the click opens the video on YouTube. Privacy

The old fear of the machine

The starting point is a widely cited and widely criticised study: in 2013, Carl Benedikt Frey and Michael Osborne estimated that around 47 percent of employment in the US faced a high risk of automation. Friesike points out that this worry is old. In the 19th century, machine breakers in England smashed looms, and according to the video, the German magazine Der Spiegel warned of mass unemployment from automation and computers in 1964 and 1978.

From the 1964 Spiegel issue, the video quotes two voices: a trade unionist who compared the hope of quickly finding new work to a man “whistling cheerfully while crossing a cemetery”, and VW chief Heinz Nordhoff, who considered technical progress in the car industry to be almost complete. Both were wrong. Occupations such as charcoal burner or music engraver have almost disappeared, but mostly work has changed, because automation only takes over parts of activities.

First: we know more than we can tell

In 1966, the chemist and philosopher Michael Polanyi observed that people perform many tasks intuitively without being able to explain the rules behind them, such as recognising emotions in faces. This tacit knowledge was long a hard limit for computers, which had to be instructed step by step. In 2014, the economist David Autor called this “Polanyi's paradox” and at the same time pointed out that machine learning works around the limit by inferring rules from examples.

Friesike concedes that the limit is shifting with AI. According to Friesike, AI recognises patterns but does not know what people know implicitly. That is why experienced people benefit most: they can judge whether a result is usable, needs reworking or should be redone from scratch. People with little experience, by contrast, accept whatever comes out, as shown by AI-written job application letters that are flawless but interchangeable. At the same time, it becomes harder to build experience when exactly the practice tasks are automated.

Second: the ironies of automation

In 1983, the psychologist Lisanne Bainbridge described how automated systems do not run by themselves: people have to monitor them, check results and step in when something goes wrong. Because the machine does the work in normal operation, these people hardly practise their skills any more. They therefore need more training, not less. The video illustrates this with a truck driver who, after years with assistance systems, suddenly has to drive without them.

Applied to AI, this means: anyone who cannot reach a usable result without AI can neither cope with an outage nor check whether a seemingly correct result is right. Ian Hannigan, Ian McCarthy and André Spicer coined the term “botshit” in 2024 for false chatbot content that people adopt without checking. Friesike sees this as one reason why AI is no simple answer to the shortage of skilled workers.

Third: expectations grow too

Technology also changes what is expected. The managing director of a large bank told Friesike that banking supervisors align their requirements with technical possibilities, so that work tends to increase. A study by Michaël Chevallier (2023) on the care robot Paro found that care workers gained no time but got a new task: persuading residents to engage with the robot in the first place.

For AI, Friesike expects something similar: time saved flows into higher expectations, more emails and denser work rather than relief. How activities change, however, is not determined by technology but by a process of negotiation in which professional groups defend their responsibilities or gain new ones; the video refers to a study by Faulconbridge, Sarwar and Spring (2023). Friesike's suggestion: use AI for tedious routines such as bureaucracy, not for the parts of work you care about.

Ideas

AI usually takes over parts of activities, rarely entire occupations.
Tacit knowledge cannot be fully captured in rules or prompts.
Judging AI results requires your own experience in the field.
Automation demands more training for the people who monitor it.
AI texts adopted without checking turn into botshit.
Time saved often disappears into higher expectations and denser work.
People negotiate how activities change, not the technology.

Insights

The video turns the usual question around: AI does not mainly threaten experienced people, it widens their lead. Those at risk are rather newcomers who no longer build experience themselves when routine work goes to machines.
Bainbridge's finding from control rooms and cockpits applies fully to generative AI: the more reliably a system runs day to day, the less often people practise the emergency in which they have to step in.
Efficiency gains are rarely paid out as free time. Whether AI relieves people or intensifies work is a question of organisation, not of technology.

Facts

In 2013, Frey and Osborne rated around 47 percent of US employment as highly susceptible to automation.
In “The Tacit Dimension” (1966), Michael Polanyi wrote the sentence “We can know more than we can tell”.
In 2014, the economist David Autor introduced the term “Polanyi's paradox” in an NBER working paper.
Lisanne Bainbridge published “Ironies of Automation” in 1983 in the journal Automatica (volume 19, issue 6).
Hannigan, McCarthy and Spicer call inaccurate chatbot content used without checking “botshit”.

References

Critique

The third argument rests mainly on a conversation with the managing director of a bank and a single ethnographic study of a care robot. The video does not back its core claim, that AI will not take away jobs in the foreseeable future, with labour market data; whether AI works differently from earlier automation therefore remains open.

Remarks

Menschen und Muster is a YouTube channel of the German public broadcaster rbb; the video dates from 30 April 2025, the script was written by Katharina Berr and Sascha Friesike.
The video reproduces the 1964 Spiegel quotes; the original issue was not consulted for this post.

Recommendations

For every use of AI, define who checks the results professionally, and do not hand this check to inexperienced staff.
Deliberately give newcomers tasks without AI so they build experience and judgement.
Practise emergencies regularly without automation, from restore tests to configuring a network by hand.
Before introducing AI, agree what saved time should be used for instead of silently turning it into more output.

Watch the video

Search the Web Archive