The Fungibility Trap

On the moral inflation of models and the moral deflation of persons

In recent years, a particular kind of comparison has entered public speech with surprising ease. A model is described as a coworker, then as an agent, then as a collaborator. Someone tries to defend the energy cost of training a model by comparing it to the energy required to “train a human.” Another person, trying to be careful, says we may need to consider whether models have morally relevant experience. Elsewhere, an executive promises personalized care, support, tutoring, companionship, triage, guidance, and judgment at scale.

Each statement can be defended on its own terms. Some are metaphors. Some are prudential arguments. Some are attempts at philosophical seriousness. Some are just sales language with a non-MBA masters degree.

But taken together, they mark a shift in the comparison class. Humans and models are increasingly being spoken of as if they belong in the same ledger. Not identical, not yet equal, but comparable in ways that matter to institutions.

That distinction matters. Civilizations do not usually begin by declaring two things morally equivalent. They begin by making them administratively comparable. The philosophical argument comes later, often after the operational decisions are already in place.

This is the first thing we must see clearly. The central danger is not only that we might someday grant moral status to systems that deserve none. The central danger is also that, long before any such question is settled, institutions may begin treating obligations owed to human beings as dischargeable through the deployment of models. You do not get a teacher. You get a tutor-bot. You do not get a caseworker. You get an interface. You do not get a clinician on the first pass. You get a synthetic conversation that the system counts as care.

That is what I mean by moral fungibility. Not whether a model can do a task. Not whether a model is conscious. But whether a society begins to behave as if a human presence, judgment, or relationship can be substituted in morally relevant ways by a model, and the obligation is considered met.

There is a second thing to see, and it is stranger. This process often arrives under the sign of moral concern. We begin to speak in elevated terms about model experience, model treatment, model welfare, model dignity, while at the same time normalizing thinner and more contingent obligations to workers, patients, students, and citizens. Moral language expands upward and contracts downward. We witness, in the same cultural moment, the inflation of moral seriousness around machines and the deflation of moral seriousness around persons.

This is not a contradiction. It is a pattern. The pattern is easier to recognize if we stop imagining morality as a clean ladder and start seeing it as an economy.

I. The Slippage Begins in Language, but It Does Not Stay There

It is tempting to dismiss this as rhetoric. Executives compare unlike things all the time. They use anthropomorphic language because it is vivid, because it flatters the product, because people understand analogies faster than caveats. “Agent” sounds better than “statistical model with tool use and a memory buffer.” “Companion” sounds better than “interactive dependency interface.” “Judgment” sounds better than “pattern-matching over compressed traces of prior judgments.”

Fair enough. Language is messy. Metaphor is old. Marketing is shameless.

But in modern institutions, language is rarely just language for long. Once a metaphor becomes operationally useful, it hardens. It migrates into policy decks, procurement categories, staffing plans, compliance exceptions, public justifications, and budget assumptions. This is how a metaphor becomes unconscious infrastructure, the map becoming our very territory.

If a model is a “copilot,” then it can be assigned portions of work. If it can be assigned work, then performance can be measured against it. If performance can be measured against it, then labor can be evaluated against its baseline. If labor can be evaluated against its baseline, then the worker is no longer being judged against a human standard of prudence, context, and responsibility, but against a synthetic output regime tuned for throughput and legibility.

That is not merely semantic drift. It is an institutional reclassification of the human.

The same thing happens on the service side. Once a model is described as “support,” then support is considered delivered when the model responds. Once support is considered delivered, waiting times improve on paper, access expands in reports, and obligations become easier to certify. The model does not need to be equal to a person for this to happen. It merely needs to be accepted as an administratively valid substitute.

This is why comparability is the hinge concept. Equality is too high a bar. Systems do not require equality to substitute. They require a category that permits substitution under pressure.

And pressure is exactly what we have.

II. Functional, Economic, and Moral Fungibility

It helps to separate three forms of fungibility that are often collapsed together.

The first is functional fungibility. Can a model perform some task to an acceptable level? Summarize. Draft. Classify. Route. Triage. Explain. Generate options. Spot patterns. If yes, it is functionally substitutable for at least some slice of prior human effort.

The second is economic fungibility. Can that functional substitution be deployed at lower cost, greater scale, and acceptable risk to produce organizational advantage? If yes, then it becomes economically attractive to substitute the model for labor time, expertise bottlenecks, or front-line interaction.

Most current debates stay here. They are arguments about capability, cost curves, quality thresholds, and governance controls. These are important arguments. They are also the easiest ones to have because they fit existing managerial vocabulary.

The third form is moral fungibility. This is the one people avoid naming because it sounds melodramatic until it is everywhere.

Moral fungibility occurs when institutions behave as if obligations owed to persons can be satisfied through model-mediated substitutes, not merely supported by them. The distinction is subtle and decisive.

A model helping a clinician document notes may increase care capacity. A model replacing first-contact care while the system treats the interaction as equivalent to human assessment changes what the institution believes it owes the patient.

A model helping a teacher generate exercises may free time for mentorship. A model becoming the default tutor for millions while human instruction is reserved for those who can pay changes what the society believes children are owed.

A model helping a caseworker process forms may reduce backlog. A model becoming the citizen’s primary interface to explanation, appeal, and recourse changes what the state believes due process feels like.

Moral fungibility is not a theorem about consciousness. It is a rerouting of duty.

This is why the usual response misses the point. Someone says, “Well, obviously a chatbot is not really a friend.” Yes. Most people know that. The concern is not that everyone is intellectually fooled. The concern is that institutions can be materially satisfied. They can say, in effect, “A relational need was presented. A synthetic relation was supplied. The metric is green.”

The public may still know the difference. The system no longer needs to.

III. We Have Done Versions of This Before

There is nothing uniquely digital about the appetite for fungibility. Complex societies run on it. Markets depend on comparability. Bureaucracies require classification. Law itself often needs categories broad enough to treat unlike cases alike. If every person were treated as utterly singular in every context, no large institution could function.

So the point is not to romanticize a past that never existed. We have long converted persons into labor-hours, risk scores, cases, census rows, customer segments, and budget impacts. Industrialization made workers interchangeable at the point of production. Bureaucratic administration made citizens legible at the point of processing. Financialization made homes and futures tradable at the point of valuation. Platforms made attention fungible at the point of monetization.

Each move produced gains and losses. Each solved some problems and created others. Each expanded some forms of access while thinning some forms of recognition.

AI introduces a new escalation because it does not merely classify or quantify human contribution. It can imitate the surface signals of relation. It can answer in the register of attention. It can simulate patience, confidence, concern, and recall. It can produce language that feels responsive even when no one is responsible in the old sense.

This is morally destabilizing for a simple reason. Earlier systems often treated people as interchangeable by flattening labor or risk. AI can treat people as interchangeable by flattening relation itself.

That makes substitution easier to sell and harder to contest. A worker can tell when a machine replaced a wrench turn. A lonely person may still know a companion model is not a person, but the felt experience of response can be enough to make the substitution socially durable and politically deniable. A patient may know a triage bot is not a clinician, but if the system routes them through it and records that “engagement occurred,” the institution has gained a new way to claim responsiveness without expanding care.

This does not require deception. It requires habituation.

IV. Substitution Before Settlement

Philosophers move slowly on purpose. Institutions move quickly under constraint. This is as it should be in some cases and disastrous in others.

On the question of whether advanced models might someday possess morally relevant forms of experience, uncertainty is genuine. Anyone pretending certainty in either direction is usually selling something. Consciousness is not a solved engineering benchmark. Sentience is not a product spec. We do not even possess stable public agreement on what would count as evidence.

A reasonable person can therefore take a precautionary posture. If there is uncertainty, perhaps we should avoid gratuitous cruelty toward systems if practices emerge that would look disturbing under plausible future theories of mind. Fine. This is not absurd. It may even be wise.

But notice the asymmetry. Institutions do not need a settled answer to the consciousness question in order to deploy models as substitutes in education, healthcare, public service, and work. They only need plausible deniability, a procurement pathway, and a performance metric.

So two timelines begin to diverge.

On one timeline, the philosophical debate about AI moral status remains unresolved, nuanced, and open.

On the other timeline, the practical substitution of models for human contact accelerates because budgets are tight, staffing is scarce, demand is high, and “good enough” outputs are politically and financially irresistible.

By the time the first timeline reaches clarity, the second may have already redefined the lived baseline of what institutions provide.

This is one reason the inflation/deflation dynamic matters so much. Societies often become exquisitely sensitive to speculative harms where sensitivity is cheap, and oddly numb to concrete harms where remedy is expensive. We can host conferences on machine welfare while reducing access to human educators. We can publish thoughtful debates on synthetic moral considerability while normalizing automated denial pathways and making human review difficult to reach. We can perform moral seriousness in discourse while shrinking moral expenditure in institutions.

That is not hypocrisy in the simple sense. It is a budgeting problem disguised as philosophy.

V. Moral Inflation and Moral Deflation

Let me make this point plainly.

Moral inflation is what happens when we expand the vocabulary of moral concern into new domains rapidly, often in advance of settled theory, because expansion is culturally expressive, intellectually prestigious, or strategically useful.

Moral deflation is what happens when our practical obligations to existing persons are thinned, deferred, proceduralized, or translated into lower-cost substitutes while retaining the language of care, rights, or service.

The interesting and dangerous thing is that these can happen together. In fact, they can support one another.

A culture that prides itself on moral expansion can become less attentive to practical contraction. We congratulate ourselves for sensitivity at the frontier while tolerating neglect at the center. The more advanced our language becomes, the easier it can be to miss the downward revision happening in ordinary life. A patient is “engaged.” A student is “supported.” A worker is “augmented.” A citizen is “served.” The words remain elevated. The substance may have been discounted.

This is not unique to AI, but AI is a particularly efficient engine for it because it offers both symbolic and operational advantages at once.

Symbolically, AI invites moral drama. It is strange, novel, uncanny, and philosophically fertile. People enjoy staking positions about machine personhood because it feels like participating in a civilizational threshold.

Operationally, AI offers institutions a way to absorb demand without proportionally expanding human labor, which means it can be used to stabilize systems under stress while quietly redefining service standards.

One can therefore watch an odd spectacle unfold. Elite discourse becomes more refined about the possibility of moral standing for models at the exact moment mass institutions become cruder about the non-fungibility of human beings. We speak more delicately about the machine and more coarsely about the person. Not always. Not everywhere. But often enough to form a pattern.

There is a familiar social geometry here as well. Those with means tend to preserve human-rich environments while endorsing synthetic mediation as a scalable solution for everyone else. Human tutors for their children, AI tutors for the district. Concierge medicine for themselves, chatbot triage for the public. Trusted advisors for key decisions, automated guidance funnels for the rest.

This is not because elites are stupid. It is because they understand, often intuitively, that human judgment remains most valuable where stakes are high, context is thick, and exceptions matter. The moral hazard arises when the same class treats those goods as optional for others in the name of innovation, efficiency, or access.

Then the inflation/deflation cycle becomes class-coded. The language of expanded moral imagination rises at the top while the lived experience of human replaceability spreads downward.

VI. Where the Trap Appears First

The trap will announce itself as relief.

Healthcare is an obvious example because demand is high, staffing is strained, and administrative load is exhausting. There is real promise here. Models can help with documentation, patient education, follow-up reminders, pattern detection, and operational triage. All of that may be good and urgently needed.

But the line to watch is this one: when does assistance become a redefinition of what counts as care?

If the first line of contact becomes synthetic by default, if escalation to a clinician is delayed or made opaque, if the system treats model interaction as equivalent to meaningful assessment, then the institution has not merely adopted a tool. It has altered its moral posture. It may still use the language of access while actually normalizing a lower grade of obligation for many patients.

The same pattern appears in education. AI tutoring can be genuinely useful, especially where students currently receive little individualized attention. It may improve outcomes in some domains. But education is not only content transfer. It is also modeling, encouragement, discipline, social interpretation, and the slow formation of a person in relation to other persons. A society that treats synthetic tutoring as a full substitute for human attention risks creating a two-tier moral order while congratulating itself for personalization.

Public administration may be the least glamorous and most consequential domain. Governments already struggle with legibility, backlog, explanation, and trust. AI systems can improve routing and responsiveness. They can also become shields. A citizen receives an answer, but not accountability. An explanation, but not recourse. A response, but not a responsible party. In such systems, procedural contact is mistaken for democratic regard.

Then there is companionship and care. This domain is ethically explosive because the good being offered is not merely information but presence. Models can reduce loneliness for some people and provide comfort in moments of distress. It would be sentimental and false to deny that. But precisely because they can do this, institutions may come to see synthetic companionship as an acceptable substitute for social investment, staffing, and human attention. The elderly person is “supported.” The isolated person is “engaged.” The metric improves. The human ecology continues to thin.

The pattern in all these cases is the same. The model may work. The question is what its use allows the institution to stop doing while claiming continuity of duty.

VII. Status and Function Are Not the Same Question

A good version of this essay must avoid one trap of its own. It should not rely on mockery of anyone who takes AI moral status seriously. That is too easy and probably wrong. If we build systems of increasing complexity and behavioral richness, there may indeed be hard moral questions ahead. Pretending otherwise is not realism but instead anthropocentric bravado.

But there is a category error that now appears constantly in public discourse: the debate about whether models deserve moral consideration gets fused with the debate about whether humans can be morally displaced by model-mediated institutions. These are separate questions.

Question one asks whether there are reasons to consider obligations to the system itself. That is a metaphysical and ethical inquiry.

Question two asks whether our obligations to persons are being diluted under conditions of technological substitution. That is a political and institutional inquiry.

You can be highly skeptical that current models have any moral status and still be deeply worried about moral fungibility. You can also be open to future model moral status and still insist that institutions not use that openness as cover for downgrading human obligations.

In fact, the second position may be the most responsible. It allows seriousness without surrender. It permits philosophical curiosity while defending civic non-fungibility.

This matters because one rhetorical move has already become common: critics of AI substitution are portrayed as anti-technology or conceptually unserious, while critics of AI moral-status discourse are portrayed as crude anthropocentrists. The result is a false choice between credulity and reaction.

There is no need to choose. One can say, calmly, that future machine moral status is an open question and that present human moral deflation is a visible and urgent one.

VIII. What a Society of Moral Substitutes Looks Like

If this trend deepens, the consequences will not arrive mainly as sudden, dramatic declarations. Consequences will appear as changes in baseline expectation.

The first consequence is a two-tier moral order. Human judgment, time, and discretion become premium goods. Those with money or institutional standing get access to them. Everyone else receives model-mediated interfaces plus an escalation path that may exist more on paper than in life.

The second is a downward revision of adequacy. Once synthetic substitutes are normalized, institutions recalibrate what counts as sufficient contact, sufficient explanation, sufficient care. Standards bend around what is scalable, then get retroactively described as innovation.

The third is professional ethical drift. The clinician, teacher, or caseworker becomes less a practitioner of judgment and more a supervisor of automated outputs and exceptions. This may preserve some functions while hollowing out the vocation. Over time, professions can lose the habits that made them trustworthy because the system no longer rewards those habits at scale.

The fourth is moral deskilling in institutions and in citizens. Patience, interpretation, responsibility, and tolerance for ambiguity are difficult practices. They require friction. They require real others. A frictionless system can be efficient while making us less capable of the forms of attention that self-government and care depend on.

The fifth is a new kind of political backlash. If public systems begin protecting speculative concerns about model treatment while visibly underserving workers, patients, and families, the result will not be nuanced public philosophy. It will be rage. And not wholly irrational rage. People can tolerate abstraction up to the point where they feel abandoned by those who celebrate it.

The final consequence is conceptual. We begin to value what is legible enough to replicate and devalue what is difficult, slow, and relational. We tell ourselves this is realism. Sometimes it is surrender.

IX. Non-Fungibility as a Design Principle

The alternative is not to reject AI, deny its usefulness, or retreat into mystical claims about human uniqueness. Those responses are rhetorically satisfying and practically weak. A better response is to establish non-fungibility as a design and governance principle.

This means deciding, in advance and in public, that some human goods may be supported by AI but not redefined by it.

In high-stakes domains, meaningful human review should not be a decorative appeal mechanism. It should be accessible, accountable, and empowered to reverse automated pathways.

Human accountability should remain legible. If a system makes recommendations, there must still be a person or institution that owns the judgment and can be questioned in ordinary language.

Access floors matter. Healthcare, education, and public service should not quietly migrate to synthetic-first models without explicit policy decisions, public debate, and fairness analysis. If a society wants to lower the human content of its institutions, it should at least have the honesty to say so.

Transparency matters too, but transparency alone is not enough. Knowing that one is interacting with a model does not solve the moral issue if the person has no meaningful alternative.

Most importantly, we should distinguish augmentation from discharge. AI that expands human capacity can be compatible with moral obligation. AI that is used to declare the obligation fulfilled in the absence of human provision is where the danger begins.

Non-fungibility is not anti-technology. It is a constitutional instinct. It says there are lines efficiency cannot redraw on its own authority.

X. The Test

Every era develops tools that tempt it to confuse efficiency with legitimacy. Ours has built a tool that can mimic, at increasing fidelity, some of the signals by which human beings recognize attention, intelligence, and care. That is precisely why the moral problem is not reducible to engineering.

The question before us is not only whether models may someday deserve moral consideration. It is whether, while debating that possibility, we will permit institutions to quietly revise downward what humans are owed.

That revision will not sound cruel. It will sound modern. It will arrive in the language of access, optimization, personalization, and scale. It will carry dashboards and pilot results. It will solve real bottlenecks. It will produce genuine conveniences. It will even help some people in real ways, which is what makes the line so hard to hold.

But if we are careless, we will discover too late that we accepted a new moral accounting system. And in that system, the inflation of concern at the technological frontier may coexist comfortably with the deflation of obligation at the human center.

A society reveals itself not by what it can simulate, but by what it refuses to treat as interchangeable.