The AI Substitution Trap

I am working through an article on AI, organizations, and labor substitution. The argument is still in progress, but the basic problem seems increasingly clear: many organizations are thinking about AI substitution as if it were mainly a question of productivity. Can AI perform the task? Can it produce acceptable output? Can it reduce headcount? Can it lower payroll costs? Those are real questions, but they are not the only ones. They may not even be the most important ones.

The deeper question is organizational: what does an organization become dependent on when it tries to become less dependent on workers?

That is the core of what I am calling the AI substitution trap. AI promises a way to reduce dependence on labor. Workers are expensive, embodied, only partially governable, legally protected in some contexts, politically contested, and capable of withholding cooperation. They possess tacit knowledge, informal routines, practical judgment, and sometimes collective power. From the standpoint of managerial control, that makes labor both necessary and troublesome. AI appears to offer a cleaner alternative: scalable, standardized, tireless, measurable, and externalized.

But substitution does not eliminate dependence. It changes its object. An organization that replaces workers with AI may reduce its dependence on employees, but it may also reorganize itself around vendors, models, cloud platforms, data pipelines, APIs, compliance systems, and technical infrastructures it does not fully control. What first appears as autonomy from labor can become dependence on suppliers. The organization cuts one dependency and creates another.

The trap becomes serious when the organization removes the very human expertise it needs to evaluate, repair, or exit the AI system. A firm may begin with workers using AI to do their jobs faster. Managers then interpret those gains as evidence that fewer workers are needed. Headcount falls. Workflows are redesigned around the AI system. Vendor tools become embedded in everyday operations. The remaining employees stop practicing some of the judgments that once made them competent. Junior roles disappear. Training pipelines weaken. The organization still receives outputs, but fewer people inside the organization can say whether those outputs are good, diagnose what went wrong, or reconstruct the work if the vendor relation breaks down.

That is not a simple automation story. It is a story about organizational autonomy.

A useful distinction is between augmentation and non-redundant substitution. Augmentation uses AI to improve work while keeping human capability intact. Workers remain practiced, knowledgeable, and able to act independently. Non-redundant substitution is different. AI becomes the primary performer, monitor, or evaluator, and the organization removes the human fallback capacity needed to perform, evaluate, or rebuild the work without the system. The issue is not whether AI performs one hundred percent of a task. The issue is whether the organization still has the human capacity to govern the new arrangement.

The working paper’s central 2x2 is useful here:

The lower-right cell is the trap. This is where an organization externalizes core work to AI vendors while also eliminating the internal capacity needed to leave those vendors. It may still have a contract that allows termination. It may still legally own itself. It may even believe it has multiple supplier options. But practical exit can become noncredible. Data may not be portable in any meaningful sense. Workflows may be built around vendor-specific systems. Remaining managers may not understand the work outside the AI interface. Former workers may be gone. New workers may never have learned the underlying task. The organization has the formal right to leave, but not the practical ability to do so.

That is what I mean by capture without ownership. The vendor does not need to acquire the organization. It does not need to own the firm. It only needs to become so embedded in routines, data systems, evaluative standards, and operating infrastructure that the organization cannot plausibly exit without major disruption. The client remains formally autonomous while losing practical autonomy.

This is different from ordinary vendor lock-in. Vendor lock-in is part of the problem, but the AI substitution trap adds capability hollowing. The danger is not only that switching costs rise. The danger is that the organization destroys its own ability to know how to switch, what to switch to, and whether the alternative is working. The supplier’s power grows not only because it controls a scarce resource, but because the client has weakened itself.

This is why “human in the loop” is often too weak as a safeguard. A human can be nominally in the loop while lacking the skill, authority, time, or organizational support to intervene. The relevant question is not whether a person appears somewhere in the workflow. The question is whether people remain competent and empowered enough to evaluate, challenge, repair, override, and reconstruct the system. A human who rubber-stamps AI output is not a governance mechanism. A depleted expert who no longer practices the underlying work is not a credible fallback.

There is an old lesson from automation research here. Automated systems often leave humans responsible for abnormal conditions while depriving them of the routine practice needed to handle those conditions well. AI can reproduce this irony in knowledge work. Organizations may expect people to supervise AI, but supervision requires expertise. If substitution removes the work through which expertise is maintained, oversight becomes theatrical.

This is especially dangerous for entry-level work. Organizations often treat junior roles as low-value labor that can be automated away. But entry-level work is also a training infrastructure. Routine cases, errors, customer interactions, drafting, debugging, document review, and supervised repair are how people acquire judgment. If AI removes those tasks entirely, the organization may preserve today’s senior experts while destroying tomorrow’s. That may look efficient inside a budget cycle. Over time, it can become developmental hollowing.

The cost argument also needs to be broadened. AI substitution is often justified by comparing visible payroll costs to apparently cheaper AI outputs. Payroll is easy to see. Licenses and model calls may look cheaper. But reliable organizational AI has system costs: cloud infrastructure, data integration, cybersecurity, compliance, validation, monitoring, error correction, model-risk management, vendor management, legal review, migration risk, and the labor required to explain or repair output. The marginal cost of generating an answer may be low. The system cost of making that answer reliable inside an organization may be much higher.

This is why early productivity gains can be misleading. AI may help existing workers do more work faster. But those gains may depend on the workers’ knowledge. If the organization treats those gains as proof that workers can be removed, it may eliminate the very competence that made the AI useful. The first stage looks like productivity improvement. The second stage becomes dependence.

There is also a managerial control premium. Managers may pursue AI substitution not only because it is more productive, but because it weakens labor claims. AI can support headcount reduction, wage restraint, anti-union strategies, reduced dependence on scarce occupations, and a symbolic claim that the organization is modernizing. The threat of AI can discipline workers before substitution is technically mature. This means organizations may substitute beyond what productivity evidence would justify, because substitution provides control advantages.

That point is important because debates about AI often assume organizations are primarily optimizing efficiency. Sometimes they are. But organizations also pursue control, legitimacy, investor approval, and competitive signaling. A company may adopt AI substitution because competitors are doing it, because consultants are selling it, because investors expect it, because executives want to appear decisive, or because non-adoption looks backward. Under uncertainty, organizations often imitate one another. If enough firms adopt AI substitution as a sign of modernization, others may follow before the efficiency case is settled.

This creates the possibility of ecological over-substitution. AI substitution spreads not because every organization has proven durable gains, but because adoption becomes a signal of competitiveness. Managers may fear being wrong alone more than being wrong with everyone else. Once roles are eliminated, routines redesigned, and vendor systems embedded, reversal becomes difficult. The organization may discover the costs later, after the human capacity needed for reversal has already weakened.

The practical alternative is not anti-AI romanticism. The point is not that AI should never replace human tasks. Some tasks are modular, standardized, low-risk, easily verified, and well suited for substitution. Some organizations will use AI intelligently while preserving internal expertise. Some will maintain portable data architectures, multi-vendor options, fallback routines, and real human evaluative capacity. The argument is not that AI substitution always produces capture. The argument is that substitution must be evaluated as a change in dependence structure, not only as a productivity decision.

The key concept here is credible exit capacity. An organization has credible exit capacity when it can switch suppliers, rebuild internal work, or continue operating during vendor disruption without catastrophic loss. That capacity has several dimensions. There is formal exit: contract rights, termination clauses, access to data. There is technical exit: portability, modular workflows, interoperable systems. There is economic exit: the ability to absorb migration costs, retraining, and temporary performance degradation. And there is human exit: retained experts, fallback teams, junior training pipelines, and enough internal judgment to know whether the substitute is working.

A contractual right to leave is not the same as the capacity to leave. That distinction is central. Organizations may believe they are not captured because they have termination rights. But if leaving would require rebuilding lost expertise, migrating deeply integrated workflows, recovering context from vendor systems, retraining staff, and accepting months of degraded performance, exit may be formal rather than credible.

This has implications for management. Procurement should not treat AI as a software purchase while HR treats it as headcount reduction. The risk lies precisely in separating those decisions. If procurement signs long-term AI contracts while HR eliminates the people who understand the work, the organization may buy short-term savings at the cost of long-term autonomy. AI governance has to include workforce planning. Workforce planning has to include vendor dependence. Neither can be understood separately.

A serious AI substitution review should ask basic questions. Which human capabilities are being removed? Who inside the organization will still know how to evaluate the work? How will junior workers learn the task if AI performs the routine cases? Can the organization operate if the vendor changes pricing, model behavior, data terms, or access? Are workflows portable? Are data usable outside the vendor system? Who can audit the outputs? Who can repair failure? Who has authority to override the system? What would it cost to reverse the decision in two years?

These questions are less glamorous than discussions of artificial general intelligence, but they are more immediate for most organizations. The near-term risk is not that AI becomes an omnipotent mind. The near-term risk is that organizations reorganize around systems they cannot govern because they have removed the people who made governance possible.

There is also a policy implication. AI substitution is usually framed as a labor-market problem, and it is one. Jobs may be eliminated. Wages may be pressured. Entry-level pathways may shrink. But it is also a problem of organizational capture, infrastructural dependence, and market concentration. If AI vendors, cloud providers, and model ecosystems become critical infrastructure for organizations that have eliminated internal capability, then worker protection and technological governance are connected. Preserving human expertise is not only a labor demand. It is a condition of organizational autonomy.

That means interoperability, data portability, audit rights, procurement transparency, competition policy, and workforce development belong in the same conversation. Public procurement could require evidence of retained human competence in critical AI-mediated functions. Organizations could be asked to demonstrate fallback capacity before replacing workers in high-risk domains. Training systems could protect apprenticeship pathways in AI-exposed occupations. These are not nostalgic protections for obsolete work. They are ways of preserving the social and organizational capacity to govern technology.

The substitution trap changes how we think about labor. Labor is not only a cost. It is also governance capacity. Workers carry diagnostic knowledge, repair capacity, contextual memory, informal standards, and the ability to recognize when something is wrong. They are not always treated well by organizations, and they do not always possess equal power. But when organizations eliminate labor in the name of efficiency, they may also eliminate part of their own intelligence.

This is the paradox I am trying to develop. AI substitution promises autonomy from labor costs, labor conflict, and labor-market uncertainty. But if substitution erodes the human capacity to evaluate, repair, or exit AI systems, the organization may become less autonomous rather than more. It may reduce payroll while increasing dependence. It may gain control over workers while losing bargaining power with vendors. It may preserve legal independence while narrowing practical independence.

The right ledger for AI substitution therefore cannot include payroll savings and productivity gains alone. It also has to include system costs, verification costs, technical debt, vendor discretion, switching costs, developmental hollowing, and the loss of credible exit capacity. The most important cost may be the hardest to see at the moment of adoption: the cost of losing the ability to leave.