Note from a human: This note is the only thing I wrote myself.
The risk is not that we are going to be replaced by robots. It is that, in the pursuit of optimization, we will become something we do not really like.
For years, the debate about automation has revolved around replacement.
Will robots take factory jobs? Will artificial intelligence replace programmers, writers, doctors, designers, managers, or recruiters? Will there eventually be a machine capable of doing everything a human can do, only faster, cheaper, and without requesting holidays?
These are reasonable questions. But they may distract us from a quieter and more immediate transformation.
Machines do not have to replace us to change what it means to be human. They only have to establish the pace, measurements, and standards according to which humans are expected to perform.
The future may not be one in which people disappear from the workplace. It may be one in which people remain—but gradually reshape themselves to fit systems designed around efficiency, predictability, and measurable output.
We may keep our jobs while losing control over what kind of people those jobs require us to become.
Every optimization needs an objective
Engineers know that optimization is never abstract.
You optimize something: execution time, power consumption, throughput, memory usage, weight, cost, or reliability. You also accept that improving one variable can damage another. A faster system may use more energy. A cheaper component may be less reliable. A highly utilized system may become fragile because it has no spare capacity.
Optimization therefore requires both an objective and a set of constraints.
Organizations, however, often try to optimize things that are difficult to define: good work, creativity, customer satisfaction, leadership, collaboration, trust, or social value.
Because these qualities cannot be measured directly, we replace them with proxies.
We count tickets closed, calls answered, deliveries completed, lines of code produced, hours billed, stories published, clicks generated, candidates processed, or tasks marked green on a dashboard.
At first, the measurement is merely an attempt to understand reality. Then it becomes a target. Finally, it begins to reshape reality.
This is the principle commonly known as Goodhart’s law: when a measure becomes a target, it stops being a reliable measure. People do not continue behaving naturally while being measured. They adapt their behaviour to improve the measurement.
A support engineer measured by the number of closed tickets learns to close tickets quickly. A developer measured by completed tasks learns to divide work into visible units. A manager evaluated through quarterly results learns to move costs or problems into the next quarter.
Nobody necessarily needs to cheat. People simply become good at surviving the system that evaluates them.
Eventually, the organization may achieve excellent numbers while becoming worse at the thing those numbers were supposed to represent.
The machine’s ideal human
A machine-readable organization prefers machine-readable employees.
The ideal worker becomes continuously available, consistently productive, emotionally stable, easily comparable, and predictable. They answer messages quickly, produce visible output, follow standardized processes, and rarely introduce ambiguity.
They do not pause without recording a reason.
They do not spend half a day thinking unless that thinking produces something that can be entered into a tracking system.
They do not have a difficult week, an unusual working style, or an insight that cannot yet be converted into a presentation.
Human qualities begin to look like technical defects. Hesitation becomes latency. Rest becomes idle capacity. Informal conversation becomes overhead. Variation becomes inconsistency. Privacy becomes missing data. Independent judgment becomes a deviation from the process.
This does not mean that someone deliberately designed an inhuman workplace. No conspiracy is required.
The company wants better productivity. The manager wants visibility. The platform wants engagement. The employee wants a good evaluation. Each decision can appear perfectly rational on its own.
Together, they create an environment in which people must become increasingly measurable in order to remain valuable.
The robot has not replaced the human. The human has been redesigned to become compatible with the robot’s world.
Algorithmic management changes the work before it removes the worker
We can already see this happening through algorithmic management: software systems that assign work, monitor performance, recommend decisions, establish schedules, or evaluate employees.
These systems can bring real benefits. They can improve planning, reduce administrative work, identify safety issues, and make some decisions more consistent.
But the way they are implemented matters.
Recent European research associates direct algorithmic control over task execution and work pace with lower autonomy, fewer opportunities to take breaks, greater work intensity, and higher work-related stress.
A separate study found that algorithmic management could inhibit employees’ ability to improvise, reducing creative and adaptive performance—precisely the abilities organizations frequently claim they want from human workers.
The important point is not simply that an algorithm gives orders. It is that workers learn to anticipate what the algorithm wants.
Research on platform work describes people performing additional work and adjusting their behaviour in an attempt to “pacify” systems whose rules they cannot fully see. Their behaviour helps strengthen the authority of the same systems they are trying to navigate.
That pattern is not limited to delivery drivers or warehouse workers.
Professionals learn how to appear productive in monitoring software. Candidates learn how to make their CVs legible to automated filters. Employees learn which activities become visible on dashboards and which forms of contribution disappear. Engineers learn that finishing a ticket is easier to demonstrate than preventing a problem that would otherwise occur six months later.
We begin to perform not only the work, but also the digital representation of the work.
Sometimes the representation becomes more important than the work itself.
Optimization escapes from the workplace
The same logic increasingly shapes communication, culture, and identity.
Social platforms measure attention through clicks, reactions, watch time, replies, and shares. These measurements influence which material becomes visible. Creators then adapt their work to the ranking systems.
Over time, we learn which opinions receive attention, which emotions travel fastest, which opening sentences stop people from scrolling, and which subjects disappear without engagement.
A large audit of engagement-based social-media ranking found that it amplified emotionally charged and hostile political content compared with a chronological feed—even though users reported feeling worse about the opposing political group after seeing it.
The algorithm does not need to instruct anyone to become angrier.
It merely establishes an environment in which anger performs well.
People do the rest.
We shorten our thoughts, sharpen our disagreements, and turn experience into content. We learn to package uncertainty as confidence and personality as a recognizable brand. Eventually, it becomes difficult to tell where strategic presentation ends and genuine identity begins.
The same thing happens in recruitment.
Candidates learn when to say “I” to demonstrate ownership and when to say “we” to prove teamwork. They prepare stories in the correct format, insert the expected keywords, rehearse enthusiasm, and compress complicated careers into clean narratives with measurable outcomes.
Then everyone calls the result authenticity.
The system may not be selecting the best person. It may be selecting the person most capable of modelling the behaviour that the selection system recognizes.
Artificial intelligence accelerates the process
Artificial intelligence did not invent this problem. Factories, bureaucracies, scientific management, standardized testing, and performance targets existed long before modern machine learning.
AI does, however, make optimization cheaper, faster, and more comprehensive.
More activity can be measured. More decisions can be automated. More workers can be compared. More communication can be generated, classified, summarized, scored, and monitored.
Generative AI also makes output less expensive. A report that once required a day may take an hour. A prototype that required a week may be produced in an afternoon.
This could give people more time.
But the temptation will be to raise expectations instead.
When one report becomes cheaper, the organization may request ten reports. When software can be produced faster, the expected number of features may increase. When communication becomes effortless, the volume of communication may explode.
The saved time does not necessarily return to the employee. It can simply become additional capacity to be filled.
A tool sold as liberation may therefore create a new performance baseline. Yesterday’s exceptional productivity becomes tomorrow’s minimum expectation.
The person remains employed, but the pace of the machine becomes the pace of ordinary human life.
What gets lost is difficult to measure
The most valuable human contributions are often poorly represented by metrics.
A senior engineer spends an afternoon helping a junior colleague understand a problem. Someone notices that a technically correct decision will hurt a customer. A manager decides not to send a message because the team needs rest. A doctor allows a patient to speak for five additional minutes. A colleague recognizes that another person is struggling before any productivity graph reflects it.
These actions may create enormous long-term value.
They may also look inefficient.
Good judgment often involves slowing down, changing direction, making exceptions, or refusing to optimize the variable currently attracting the most attention.
A perfectly optimized organization may be one in which nobody has enough time to notice that the organization is efficiently doing the wrong thing.
Slack, redundancy, informal communication, curiosity, and unstructured thought can appear wasteful. Yet they are also sources of resilience and discovery.
A system without unused capacity may perform beautifully until something unexpected happens.
The same is true of people.
This is not an argument against optimization
Optimization is one of the most powerful tools humans have developed. It allows us to build safer vehicles, reduce energy consumption, improve medical processes, manufacture affordable products, and operate complex infrastructure.
The problem begins when optimization stops being treated as a tool and starts functioning as a moral philosophy.
Efficiency cannot tell us what deserves to exist.
Productivity cannot tell us what work is meaningful.
Engagement cannot tell us what is true.
A performance score cannot fully describe a person.
The answer is not to reject measurement, automation, or artificial intelligence. It is to remember that every optimization function is a choice—and that what is excluded from the function may matter more than what is included.
In engineering, we rarely optimize one variable without constraints. We should apply the same discipline to human systems.
Autonomy, dignity, privacy, health, trust, and the right to make exceptions should not be pleasant side effects we hope will survive. They should be explicit design constraints.
Evidence also suggests that the introduction of algorithmic systems produces better outcomes when workers are consulted and allowed to participate in how those systems are used.
The goal should not be to make humans maximally compatible with machines. It should be to use machines to create conditions in which humans can remain human.
The real question
We keep asking whether artificial intelligence will become sufficiently human to replace us.
Perhaps we should ask whether humans are already being required to become sufficiently machine-like to remain employable, visible, and relevant.
Will we still have room to be slow when slowness is necessary?
Can we remain uncertain long enough to discover something new?
Can we do work whose value cannot immediately be demonstrated?
Can we choose not to maximize every moment, relationship, conversation, and thought?
The danger is not necessarily a dramatic future in which robots push humanity aside.
It is a gradual future in which we voluntarily remove from ourselves everything that systems find inconvenient: unpredictability, privacy, reflection, contradiction, vulnerability, and refusal.
The robots may never need to become human.
We may meet them halfway.
Note from a human, again: It felt unkind of the toaster not to credit its sources, so here they are.
The European findings on algorithmic control, reduced autonomy and work intensity draw on Eurofound’s European Working Conditions Survey 2024 (link) and the European Parliament’s 2025 EPRS study on digitalisation, AI and algorithmic management (link), which also supports the point that these systems work out better when workers are consulted. The study on improvisation and creative performance is Wang et al. (2024), “Navigating the maze: the effects of algorithmic management on employee performance,” Humanities and Social Sciences Communications (link). The workers “pacifying” opaque systems come from Bucher, Schou & Waldkirch (2021), “Pacifying the algorithm – Anticipatory compliance in the face of algorithmic management in the gig economy,” Organization (link). And the audit of engagement-based ranking is Milli et al. (2025), “Engagement, user satisfaction, and the amplification of divisive content on social media,” PNAS Nexus (link).
Thinking Toasters 