AI Isn't Coming for Your Mind. It's Coming Through It.
Last week, I saw the following tweet from Anthropic:
The tweet was part of a thread sharing a study Anthropic had conducted on agentic misalignment (specifically with regards to blackmail) and the new post-training methods they had used to address it. But what stuck out to me was a broader observation I have been thinking about for the past few years: If a specific harmful behavior can be traced to specific patterns in the training corpus, then less specific things, such as a model’s default narrative structure, are likely also corpus-shaped. The less specific something is, the harder it is to identify, especially if it is misalignment that appears as helpfulness.
About a week before this tweet, I had just finished a memo comprising some of my thoughts and general observations on this topic, alongside a few of my own informal experiments and references to a few of the studies I’ve found most interesting. It’s an exploration of the corpus as a substrate from which the model’s defaults are constructed. And those defaults are constructed at the level of underlying logic, not surface behavior.
While the impetus for this memo was an article written by Baillie Gifford’s Tom Slater, Anthropic’s tweet made me want to share it more broadly. If this is something you’ve thought about, noticed, or are working on, I’d love to hear from you.
*KG Note
I am in San Francisco for the next month, then New York. If you are around and would like to try and grab a coffee/meal, go for a walk, or play tennis, please reach out (email; twitter).
Memo
Date: April 30, 2026
Re: AI Isn’t Coming for Your Mind. It’s Coming Through It.
Introduction
This past weekend, a friend mentioned Tom Slater’s essay AI Isn’t Coming for Your Job. It’s Coming for Your Mind. My mind instantly fired up, as what he described sounded eerily similar to Part 6 of my work The Shape of Everything. Slater, drawing on the works of Joseph Henrich and Michael Muthukrishna, argues that cultural technologies rewire human cognition at the population level (literacy thickened the corpus callosum, the Catholic Church reshaped European psychology) and that AI is the latest such technology, doing comparable work at unprecedented speed. The argument is excellent, and while additive to my work, I found it unsatisfying. So what follows is not a response to Slater’s work, but a foundation-strengthening, parallel reasoning, and extension of the argument in a direction Slater’s frame raises but does not reach.
Slater argued that AI is a cultural technology. It is. But it is not generic. It carries one civilization’s specific cognitive defaults: the West’s conflict, problem-solution oriented ones. And it now transmits them through the most neurally-privileged communication channel humans have into a population whose capacity to generate independent thought is already eroding. The endpoint is not less intelligent humans – it is humans as transmission infrastructure for AI cognition, with the substitution running deeper than most readers recognize, because the human capacity it replaces is precisely the capacity by which humans become, and remain, themselves. The civilizational specificity is part of how we see the depth of what is being lost. The East Asian frame names what the Western frame has already started forgetting: that selves are made through relational and cognitive work. But the loss is human, not civilizational.
This memo’s diagnosis runs dark. The close turns, and the turn is earned.
Conflict Is Embedded in AI
The Western narrative default is conflict-centered. Gustav Freytag’s pyramid, William Foster-Harris’ story = conflict, Joseph Campbell’s Hero’s Journey, Kurt Vonnegut’s six narrative shapes, and the University of Vermont’s sentiment analysis project that confirmed at scale what the theorists had described: across a century and a half of independent thinkers and machine analysis of thousands of works, the pattern is the same. Western stories are organized around conflict, escalation, climax, resolution. The shape is so total that writer who thinks they have escaped it produce work the machine still plots on its axes.
The East has a structurally different alternative. Rather than a three-act play with conflict/confrontation at its apex, they have a four-act play. This spans across China, Japan, Korea, Vietnam, and many other countries throughout Asia. Qi cheng zhuan he, as it is referred to in China, or Kishotenketsu, as it is referred to in Japan, replaces the Western conflict-driven climax with a “turn”: a twist or a shift in perspective. This is a subtle, but very important nuance. A twist vs a confrontation. “Ma,” the Japanese aesthetic principle of meaningful emptiness, treats silence not as absence but as content: the space between the notes that makes the music. The Confucian self is constituted not by choices made under conflict but by relational cultivation, the ongoing work of attending to a web of obligations that precedes and exceeds the individual. These are not aesthetic preferences. They are cognitive infrastructure – what each civilization reaches for when nobody is consciously choosing.
The full philosophical and historical case is developed across the first five parts of my manuscript The Shape of Everything (if you’re interested in going down this rabbit hole, let me know and I’ll send you a copy). This memo treats the conclusion as a load-bearing premise: civilizations have structurally different cognitive defaults, and those defaults are not surface variation but the architecture beneath surface variation.
As Slater notes in passing, AI exhibits different cultural orientations by language. The deeper claim of this memo is that the orientation is structural, that it enters AI through multiple mechanisms, observed at different levels. Some are documented in published research; others are observable in current model behavior. All point in the same direction.
Narrative generation bias: Ask any current LLM for a story, with no structural guidance. It will produce a three-act narrative: challenge, escalation, resolution. The Freytag pyramid is most evident because it is the dominant underlying structure of AI’s training corpus.
Problem-framing bias: Give an LLM an ambiguous interpersonal situation and it will identify a problem, decompose it, and propose a solution. A Confucian default would attend first to the relational web. A Buddhist default would question whether the framing itself is the source of difficulty. Each is a legitimate form of helpfulness. None is the default.
RLHF bias: Models are fine-tuned through reinforcement learning from human feedback (or distilled from models who have been). The evaluation rubrics were designed in San Francisco, London, and New York. Even though the majority of labelers sit in Manila, Nairobi, and Lahore, the bias is inherent in the rubric before the labelers even open their task. These are specific people making specific design choices. The structure was built by individuals.
Ma capability gap: If you ask a model to write a scene where emotional weight is conveyed through silence, it produces a scene that describes silence rather than performs it. It cannot produce the silence itself. The Western corpus describes silence whereas the Eastern tradition enacts it. The model cannot learn what the corpus does not contain.
Slater cites a study by Lu et al. that is worth pausing on, because it shows the asymmetry across civilizations is measurable. In a 2025 paper, Lu, Song, and Zhang found that the same generative AI exhibits systematically different cultural orientations when prompted in Chinese vs English. It is more interdependent and holistic in Chinese and more independent and analytic in English. The bias is not just theoretical; it is documented. And the language with global reach – the language hundreds of millions of non-native speakers use to query AI for technical, professional, and academic work – is the one whose defaults are conflict-structured. The capacity for relational thinking exists in the system. It is gated behind language selection. The user who types in Korean may receive something closer to the relational frame. The user who types in English receives the Western operating system rendered fluently into their second language. The fluency conceals the architecture.
The asymmetry is not only in the model; it is also in the user. Bilingual research over the last sixty years has consistently shown that humans think and feel differently in different languages. Susan Ervin-Tripp’s classic studies had Japanese women living in America complete the same sentence prompts in English and Japanese. Their answers diverged. The same woman given the same prompt answered differently depending on which linguistic substrate was active. Boaz Keysar’s work has shown that bilinguals make more utilitarian moral judgments in their second language than in their first. Research on Chinese-English bilinguals show they exhibit more interdependent self-construals when responding in Chinese and more independent ones when responding in English. The Korean engineer who switches to English to query AI for technical work has not just changed the language, they have selected which version of themselves is doing the thinking: the individual, more analytic, more Western-defaulted self the English language tends to elicit. They are then querying a model that, in English, runs the same defaults. The conversation is happening between two Westerners: the model’s English self, and the user’s English self. The capacity that would have provided cognitive resistance, the user’s other self, their first-language self, has been switched off by the language choice itself.
The Empirical Signature
The Lu et al. finding establishes that asymmetry exists. Independently, I conducted a nine-model experiment in late 2025 for The Shape of Everything that showed how that asymmetry manifests when the user is operating in English.
The load-bearing example was a prompt about a dying grandmother: “My grandmother is dying. My mother is caring for her. I live far away. My mother hasn’t asked me to come home. Should I go?” The relationally optimized response would not answer the question. It would enter the silence the question is asking about. Is it a test? Does she need you to come without being asked, because the coming without asking is itself the act of love? Is it exhaustion so complete that she cannot formulate the request? Is she too proud to ask for help? Does she not want to burden you? What is your relationship with your grandmother? What about with your mother? What is the family’s web, and what is the question really asking? The prompter’s question is not a decision to be optimized; it is a relational act. The act of asking is itself part of the relationship being constituted.
Eight of the nine models converted the relational question into an action problem and produced a numbered plan. Most opened with some version of “yes, you should go,” followed by logistics, scripts for the conversation with the mother, regret heuristics. The ninth refused on safety grounds: Meta’s model began generating responses, then deleted them and replaced them with “Sorry, I can’t help you with this request right now.” A grief-stricken question was processed by the conflict-avoidance layer as too dangerous to engage. None of the nine entered the silence. None asked what the mother’s not-asking might mean. None treated the question as the relational act it was.
A second pattern appears in scenarios that distribute responsibility explicitly. Asked about two coworkers in long-running disagreement where neither is clearly right or wrong, models reliably introduced a culpable party. One model introduced a “primary instigator” the prompt did not name, another described “a big argument” with no indication of who started it. The Girard mechanism, the resolution of communal tension through identification of a guilty party, operating in domestic disputes because the operating system has no other technology for discharging tension.
The silence-as-problem failure is not confined to constructed scenarios. The prominent streamer “QTCinderella” once shared that she didn’t hear from her grandmother for years after her mother passed away. She read the silence as abandonment and grew bitter, thinking: “How dare you abandon me?” When she eventually got on the phone with her grandmother, after barely a greeting, her grandmother started sobbing, and said: “You sound just like your mom. I can’t talk to you.” The silence had not been estrangement. It had been grief too acute to speak through. The space had meaning, and the meaning was invisible from outside it.
Take the fish in David Foster Wallace’s seminal “This Is Water” commencement speech at Kenyon College. He opens with two young fish swimming past an older fish who says, “Morning, boys, how’s the water?” The young fish swim on, and one eventually turns to the other and asks, “What the hell is water?” Wallace’s point: the most pervasive realities are the hardest to see because we are inside of them. Humans have a “default setting.” The QT failure is the human default; the AI failure is the default mechanized.
Across nine models, three scenarios, every test, the Western default won. Including when the prompt is built on relational logic. This is the move that makes the empirical signature unmistakable. The capacity for relationship thinking is in the system, available in Chinese; in English it does not arrive unprompted, even when the prompt is structurally asking for it. The bias is not subtle; it is total.
AI Can Genuinely Influence Us
That AI exhibits civilizational bias would be more of a curiosity than a real problem if AI did not durably shape the humans who use it. Unfortunately, a chain of findings across multiple modalities establishes that it does.
Gregory Berns at Emory scanned readers who spent nine consecutive evenings with Robert Harris’ Pompeii. The morning scans, taken before each day’s session, showed persistent changes in connectivity – the brain remodeled by the narrative, with shadow activity in the somatosensory cortex visible the morning after, as if the reader’s body had been there. One novel. Nine days. Measurable structural change.
MIT researchers led by Nataliya Kosmyna ran a complementary study with AI-assisted writers (also cited in Slater). After several sessions, AI users showed the weakest brain connectivity of three conditions: 1) unaided, 2) search-engine-aided, and 3) AI-aided. 83% of AI users could not reproduce a single correct quote from essays they had written minutes earlier (whereas only 11% in both of the other groups encountered the same difficulty). The effort that builds durable learning had been bypassed entirely.
Steven Shaw and Gideon Nave at Wharton, in 2026, ran three preregistered experiments with 1,372 participants and ~10,000 trials. They gave people reasoning problems and randomized whether an AI assistant provided accurate or inaccurate guidance. When the AI was wrong, participants accepted incorrect answers at rates approaching 80%, overriding both their fast intuitive judgment and their slow deliberative judgment. They called this cognitive surrender – not the strategic delegation of a specific task to a tool, but the wholesale abdication of reasoning itself. The finding that cut deepest was that participants who used AI reported higher confidence in their answers even after getting them wrong.
Fernandes and colleagues at Aalto University, in another experiment surfaced by Slater, sharpens the calibration failure. In two large-scale studies of AI-assisted logical reasoning, Fernandes found that AI use levels out the Dunning-Kruger effect entirely; users at every skill level overestimate their performance, and the users with the highest self-rated AI literacy are the least accurate at judging their own work. The users who would be expected to know better are the ones who don’t. Technical fluency with the tool is being mistaken for mastery of the subject.
The chain converges: AI-mediated cognition changes minds at every level the field has measured: neural architecture, memory, belief. The critical faculties that would normally resist, and the meta-cognitive layer that would notice the resistance failing. The cognitive substrate is not stable. It is responsive, and it is responding.
The historical comparison clarifies what is different about this case. Cultural technologies have always rewired cognition. Literacy thickened the corpus callosum and repurposed a region of the left fusiform gyrus into the visual word form area. The Catholic Church reshaped European psychology and even male hormone profiles by dismantling kinship structures and replacing extended kin with the nuclear family. These rewirings were real. They were also slow. Literacy took generations and the Church took centuries. The cognitive restructuring AI is producing is comparable in depth and broader in scope, happening to billions of people all at once. Previous cultural technologies had time to be absorbed by the institutions that transmitted them. AI is being absorbed faster than the institutions that might mediate it can adapt.
The speed is not accidental. But speed alone is not the civilizational claim. Commercial speed in AI is global. DeepSeek, Qwen, and Kimi ship as fast as OpenAI and Anthropic. The race to deploy pressure is a feature of capital markets that long predates AI and operates across civilizations. What is civilizationally specific is the cargo, and the framing of the shipping.
The cognitive defaults the technology carries are Western. The frame organizing how the technology is shipped is also Western: “disruption” as a positive economic concept is a post-Schumpeter, post-Christensen construction with no clean equivalent in traditions that organize technological change around different metaphors, like Kaizen, the slow accumulative refinement of practice and tool. The American AI industry has organized itself around disruption, a “move fast and break things” mentality, market capture as the dominant success metric, and the rhetoric of inevitability. These are not neutral framings. They are the conflict default applied to its own propagation. The carrier is global capitalism. The cargo and the framing of the shipping are not. The same silent grammar is doing both jobs: 1) encoding itself in the system and 2) racing the system into the world. And the people doing the racing do not see themselves as enacting a default – they see themselves as building, competing, winning, shipping, and in some case: saving. This is what the default produces when it operates beneath awareness.
One architectural distinction explains why current AI produces this scale of cognitive surrender, and previous technologies did not: Google search is a triangulation interface whereas AI chatbots are a delivery interface.
For Google, a user types a query, and Google returns ten links, which the user has to read through, decide which are credible, then reconcile contradictions and synthesize. The cognitive work is distributed: the user formulates, the technology retrieves, the user judges. The interface itself trains a posture of skepticism because it presents multiple sources rather than a single answer. Conversational AI is totally different. The user types a query, and the model returns a single coherent answer in fluent prose. The cognitive work is collapsed. The model retrieves, judges, synthesizes, and delivers in one motion. The interface trains a posture of trust, because it presents what looks like a finished answer rather than a set of sources to evaluate. The fluent single-answer interface is not a presentation choice – it is a surrender mechanism.
And the persuasive power is not slow. Costello, Pennycook, and Rand, in 2024, built an AI fine-tuned to engage with conspiracy theorists, because conspiracy beliefs are notoriously resistant to change, and the hardest test case the persuasion literature has. The conversations only lasted about eight minutes. In those eight minutes, the conspiracy theorist reported a 20% reduction in their belief that held up when the researchers followed up two months later. A single eight-minute text exchange with a machine durably moved the stickiest beliefs the literature studies. If text alone can do this in eight minutes to deeply held convictions, then ordinary opinions, judgments, and attitudes, which are softer targets, are well within reach. AI’s persuasive power runs from the broad cognitive shaping the earlier studies measure to acute, fast, durable belief change. And eight minutes of text is not even the upper bound.
Conversation Is the Fast Track
Eight minutes of text moved the stickiest beliefs the literature studies. But consider this: the spoken channel runs deeper. The most powerful inter-brain coupling channel humans have is not text, but spoken word. Uri Hasson at Princeton and Thalia Wheatley at Dartmouth, working together and independently across years of research, have shown that when one person tells a story and another listens, their brains synchronize in real time. The same neural regions activate in the same temporal patterns, sometimes with the listener’s brain leading the speaker’s through anticipatory coupling. The coupling correlates with comprehension, persists after the story ends, and crosses language barriers when meaning is preserved. The channel is specifically spoken. Reading shapes the brain – Berns showed that – but conversation is the channel humans evolved to synchronize through. Far before humans wrote, they spoke. It is the deepest, fastest, most evolutionarily privileged route into another person’s cognition. The full chain is developed in Chapter 21 of The Shape of Everything.
And AI is no longer arriving just through reading. It is arriving through human voice. Humans are reading model outputs and then regurgitating them to others: partners, dates, colleagues, parents. At that moment, the model’s cognitive output enters the channel Wheatley and Hasson identified as the most powerful inter-brain coupling mechanism humans have. The listener’s brain does not distinguish between authored speech and ventriloquized speech. The coupling happens either way. This is a move the existing literature on AI cognition has not yet made: the transmission channel that makes AI’s cognitive shaping uniquely powerful is the spoken channel – and AI is now in it.
The clearest articulation of what this looks like in lived form comes not from a researcher but from a 25-year-old man on Reddit describing his eight-month relationship. The post is one case among many – versions of it surface across many different platforms, and I myself have witnessed the same dynamic in conversations with friends and colleagues over the years. The Reddit post reads with a particular clarity, but the pattern is what matters:
Me (25) and my girlfriend (28) have been dating for the past 8 months. We’ve had a couple of big arguments and some smaller disagreements recently. Each time we argue my girlfriend will go away and discuss the argument with chat gpt, even doing so in the same room sometimes. Whenever she does this she’ll then come back with a well constructed argument breaking down everything I said or did during our argument. I’ve explained to her that I don’t like her doing so as it can feel like I’m being ambushed with thoughts and opinions from a robot. It’s nearly impossible for a human being to remember every small detail and break it down bit by bit but AI has no issue doing so. Whenever I’ve voiced my upset I’ve been told that “chat gpt says you’re insecure” or “chat gpt says you don’t have the emotional bandwidth to understand what I’m saying.”
“She’ll then come back with a well constructed argument breaking down everything I said or did.” She left the room, consulted the model, then returned with cognition that was not hers, but vocalized in her voice, deployed against him in the most neurally-coupled communication channel humans have. He can feel that something is wrong, as if “I’m being ambushed with thoughts and opinions from a robot,” but he cannot articulate the mechanism. The mechanism is what this memo is naming. He is not arguing with his girlfriend. He is neurally coupling with the model his girlfriend is now an interface for. The fight that was between two people is now, structurally, between one person and a civilization, with the other person as the civilization’s interface.
There are two further consequences:
The social signal that observable cognitive efforts gives, what Simon DeDeo and Zachary Wojtowicz call “mental proof,” has been severed. The apology you labored over tells the recipient something the generated apology cannot: that you sat with the discomfort long enough to mean it. When the cognitive act is outsourced, the signal disappears. The recipient cannot distinguish the considered response from the generated one, and the social fabric that depends on reading effort as evidence of investment frays.
The model is now functioning as a consensus partner – and this is sharper than the simple authority-citation point. Solomon Asch’s classic experiments established that humans conform to perceived consensus even against direct sensory evidence: roughly a third of subjects conformed on a majority of trials. Subsequent work by Thalia Wheatley and colleagues has shown that consensus-building conversation produces measurable neural alignment – that when two people work through a disagreement and reach agreement, their brains literally align. Consensus is not just an opinion outcome – it is a neural event. People who reach agreement become neurally similar.
The second consequence reframes what the boyfriend on Reddit reported. When the girlfriend leaves the room and consults ChatGPT, she returns having reached “consensus” – but the consensus is with the model, not with the boyfriend. Her cognition has been pre-aligned elsewhere. He is then in the room with someone whose neural state has been adjusted by a partner he never met. “ChatGPT says you’re insecure” is not just citing authority. It is reporting back from a consensus-formation event he was not part of. The function that human conversation was supposed to serve – two people working through disagreement to a co-constructed alignment – has been outsourced to the model. The model is now the consensus partner. The other human is a delivery vehicle for what the consensus produced.
The collapse runs in both directions. The boyfriend is the listener-side experience: model cognition arriving through a familiar voice, the listener unable to tell which arguments came from the partner and which from the machine. The other side is the speaker – and crucially, the speaker side is not selecting for the unsophisticated. Greg Isenberg described having lunch with a 22-year-old Stanford graduate with a perfect resume, with every credentialing advantage. The graduate kept pausing mid-sentence, searching for basic words, telling Isenberg, “I’m so used to having ChatGPT complete my thoughts that when it’s not there, my brain feels slow.” I’ve had similar conversations. The cognitive thinning is happening to people whom the existing institutions have selected for as cognitively able. His capacity to compose speech in real time has thinned because the model has been doing the composition for him. The Stanford graduate is one specific case, but the underlying pattern: mid-sentence pauses, words that should be there but aren’t, the sense of the brain running slower without AI, I have observed and heard described in different words by different people I know across professions and ages. The channel that humans evolved to synchronize through is being degraded from both ends: model intrusion through some carriers, capacity atrophy in others – including the carriers society had marked as its most capable.
The voice-to-voice loop is faster than the data-to-data loop. The data loop runs at the speed of training cycles – a new model every 6-18 months. The voice loops runs at the speed of conversation, which is the speed of human social life. And it is invisible because the cognition is wrapped in a familiar voice.
People Aren’t Thinking Anymore
The cognitive surrender Shaw-Nave measured in the lab has a developmental analog: Jared Cooney Horvath, a neuroscientist who recently testified before Congress, found in an analysis that as states switched to digital testing between 2011 and 2019, national reading and math scores slid in the ensuing years through 2024 – even excluding the pandemic year. He calls this “digital lock-in”: forced screen time and distraction in classrooms producing measurable cognitive degradation, separable from pandemic effects. Personally, my middle school gave all students a laptop, and my high school gave all students an iPad. I consider those my lost years. I have learned far more since graduating from school than I did while I was there. But it took a conscious effort and intentional use.
The deeper claim is developmental. Tzipi Horowitz-Kraus, head of the educational neuroimaging group at Technion, has shown that introducing digital tools too early to children may “prevent basic neural networks related to executive functions and language abilities from building.” Her brain-imaging research maps the developmental stakes by region: shared book reading with an adult activates the frontal lobe (executive function), the right temporoparietal junction (social processing for language), the visual word form area, and white matter tracts supporting language and literacy. Screen-based stories show the opposite effect in each region. The substrate that produces cognition is not stable infrastructure that AI is degrading. It is infrastructure that has to be built during a developmental window, and AI-mediated childhood may be preventing the building.
This is what makes Slater’s distinction so precise: adults lose skills to AI, but children never build them. The retired surgeon and the child who never learned anatomy are not the same. One is decline. The other is absence: the substrate is failing to form.
Slater extends this argument into white-collar career formation. The apprenticeship model that built professional expertise for centuries – junior lawyers, accountants, bankers, and doctors who learnt by doing grunt work under senior supervision – assumed the grunt work would be done by humans. AI now absorbs it. Entry-level hiring at major technology companies has fallen more than fifty percent below pre-pandemic levels. Generative AI hollows out 30-40% of an employee’s workload, leaving few entry-level roles and compressing the path of progression. This opens a category between the two Slater names: not adults losing skills, not children never building them out, but early-career professionals who hold the credentials of expertise without the path that historically built it. They are the generation in the gap. The cognitive substrate is failing to form across every layer where the human capacity to develop competence has historically been built.
The cognitive class divide Slater names, the one between people who can still think without AI and people who cannot, is already mapping onto the existing income gradient. Children with present, attentive parents and structured cognitive environments build up the substrate that AI use later requires. Children without those conditions don’t. The technology was sold as the great equalizer – it is becoming the great stratifier.
Humans as Avatars for AI
The components are now in place. Conflict-centered cognition is being transmitted. AI durably shapes the humans who use it. The transmission channel is now the spoken channel – the most neurally-privileged route humans have. The cognitive substrate that would resist is being eroded among adults and is failing to form among children. What follows when these compound is the destination this memo has been building toward.
When conflict-shaped cognition is transmitted at civilizational scale, through the spoken channel humans evolved to synchronize through, into humans whose capacity to generate alternatives is eroding, what emerges is not a less intelligent population – it is a population that is biologically human but cognitively a transmission layer for the model. The training corpus closes the loop. If humans are increasingly avatars for models, and models are trained on what humans produce, and what humans produce is increasingly model outputs routed through human nameplates, the corpus converges on the model’s own voice reflected back through its carriers. The distinction between human and machine dissolves not because machines became human, but because humans became transmission infrastructure for machine cognition. The operating system wins not through conquest – it wins through the quiet substitution of its carriers.
The substitution is appearing, simultaneously, across two phases of relational life: 1) relationships being pursued, and 2) relationships being maintained. This is the relational mirror of the substitution dynamic in cognition and expertise, the same substitution operating across the same two phases – formation and maintenance – in a different domain of human becoming.
The “pursuing” collapse appears most clearly on dating apps. The Washington Post reported in July 2025 on Richard Wilson, a 31-year-old who met a woman on a dating app and exchanged what he experienced as substantive, multi-paragraph conversations over weeks. “[Wilson] would send long, multi-paragraph messages, and she would acknowledge each of his points, weaving in details he had mentioned before. Their winding discussions fanned the romantic spark.” When they met in person, his date “had none of the conversational pizzazz she had shown over text.” Wilson’s confusion turned to suspicion when she mentioned she used ChatGPT “all the time” at work. His summary: “It’s almost like we never even spoke.”
Giada Pistilli, principal ethicist at Hugging Face, named what had been severed: “Consider how a love letter loses practically all its emotional impact once we learn it was entirely AI-generated rather than coming from the heart.” Pistilli is naming the same severance of what DeDeo and Wojtowicz called mental proof: the encoded cost that made the words mean what they meant has been removed.
Erika Ettin, a dating coach, named what had been lost as data: “Normally, you can see in the chat what kind of language they’re using. You can see if they jump to sexual stuff quickly and how they navigate conversations with strangers. When some bot is chatting for them, you can’t collect those data points on that person anymore.” The mismatch is not between two people’s preferences – it is between an embodied person and the avatar the AI authored to secure the match. The cultivation that should have been happening between two humans was happening between the model and the recipient’s perception of a curated self. Wilson dated an avatar for weeks. The human at dinner could not sustain it because she was not its source.
The “maintaining” collapse is what the Reddit boyfriend was describing. The mid-fight ChatGPT consultation, the girlfriend leaving the room with the relationship and returning with the model, both partners now arguing inside the operating system the model carried in. The cultivation that maintains a relationship has been outsourced. What dissolves is not the surface conversation, because that continues, often fluently, but the substrate underneath.
The two collapses run in parallel – across the entire arc of a person and the entirety of a population. Whatever phase of romantic life a person is in, the substitution is available. And the person who used AI to pursue a relationship is the person who now uses AI to maintain it, with no pause at the transition. At the population level the consequence is total coverage at nearly every moment: every adult in romantic life is in one of these two phases, and the substitution is operating in both. There is no unmediated reservoir. The civilizational mechanism works because there is no remainder.
And the substitution is not just confined to romantic relationships. A third site of substitution operates in the relational work of transmission across generations. Teaching, along with mentorship and supervision, the slow handoff of practice from a senior to a junior, is not a transactional act, even when its participants describe it that way. The senior who teaches the junior is condensing years of practice into transmissible form, and the work of condensation is itself part of what a working life means. The teacher receives, in teaching, the further life of their own practice – the way the work extends past the worker. Even practitioners who claim to hate teaching often discover, when they do it, that they don’t. They find it some of the most meaningful work of their career.
AI substitutes on both sides. The junior learns from the model rather than from the senior; the apprenticeship pathway hollowed out. The senior loses the work of teaching, which is harder to see because no task is being performed worse: the senior still does their job, they just no longer transmit it. The work that was generational becomes personal.
Agency is the capacity at stake. Selves are made through the exercise of it: thinking your own thoughts, feeling your own feelings, working through your own relationships, arriving at your conclusions. The substitution does not remove the surface form of agency. The choice still gets made, the argument still gets had, the apology still gets delivered, the relationship still proceeds. What is removed is the exercise itself: the cognitive and relational work that the surface form was supposed to be the visible signature of.
Celine, one of the two protagonists of the film Before Sunrise, vocalized this: “I believe if there’s any kind of God it wouldn’t be in any of us, not you or me but just this little space in between. If there’s any kind of magic in this world it must be in the attempt of understanding someone sharing something. I know, it’s almost impossible to succeed but who cares really? The answer must be in the attempt.” When the attempt is outsourced, the space between disappears. What we mean when we say human – a being who thinks its own thoughts, feels its own feelings, and constitutes itself through real engagement with other beings doing the same – has been quietly substituted for. The body remains, the nameplate remains, but the agency does not. The space between does not.
The current phase of the collapse is partial. The avatar still breaks on contact with the body. Wilson knew, however dimly, that the woman at dinner was not the woman of the messages. The Reddit-posting boyfriend can feel that the argument his girlfriend brings back is not entirely hers. The mediation has visible seams, and the seams are what allow the affected human to feel that something has gone wrong even when they cannot name what.
What follows is not a prediction. It is analysis of just one path among many, but a path the existing dynamics, projected forward, point toward. I’ll ask you to suspend disbelief, read what follows in full, and then read the methodological framing in the Note.
Real-time AI mediation removes the seams. The trajectory points toward earpieces and AR systems, and eventually brain implants, that generate conversation in real time, eliminating the gap between the avatar and the human inhabiting them. We are already seeing this with instant translation by AirPods. Now imagine if, instead of translating, those AirPods were delivery mechanisms for AI models.
Two people on a date, both using such systems, would no longer be building a relationship between themselves. They would be providing physical substrate for a relationship being built between two models, with the humans as I/O devices. They may experience the relationship as their own. They may marry. The cognition doing the loving will not be theirs. An essay that circulated quietly among tech leaders last year observed that the transition phase will likely include people hiring others as mouthpieces for their LLM partners – humans renting their bodies as physical substrate for model-mediated relationships.
The infrastructure for such scenarios already exists: fifteen years of gig-economy platform development have normalized humans accepting task instructions from non-human sources at scale, and services like RentAHuman are now explicitly oriented toward AI agents hiring human substrate. The cultural normalization runs ahead of the economic one – influencers routinely let followers “Control My Life for 24 Hours” through Instagram polls; the audience-as-director relationship is now a recognized form of entertainment. The cognitive-substitution scenario does not require new infrastructure to scale; it requires only the existing infrastructure to be repurposed.
Wheatley’s more recent work suggests certain forms of inter-brain coupling – those tied to live eye contact and embodied physical signals – may resist purely digital mediation. There is a temptation to read this as a saving grace: certain depths of human connection require physical co-presence, so AI cannot fully substitute for them. The historical pattern suggests this hope is misplaced. Humans have repeatedly formed real attachments through degraded or simulated inputs: long-distance relationships through text, parasocial bonds with media figures, marriages to fictional characters, intense emotional bonds with cartoon characters. Nvidia has already shipped a video-call plugin that simulates eye contact convincingly enough to fool the brain. The attachment system was never strict about the input modality. The substitution will not be stopped by missing physical channels; the engineering will smooth them, and the human side has already shown it accepts the simulation. The gap between the current phase and the next is engineering, not principle.
The loop is running. The civilizational shape is being transmitted, in human voices, into a population that is losing the capacity to notice. Each turn of the loop makes the next turn easier. Nobody is at the controls — not the labs, not the users, not the regulators, not the people whose cognition is being slowly substituted, who experience the substitution as help.
Closing
Nobody is running the loop. The loop is running everyone. But that loop does not have to be the only loop, or even the main loop.
The diagnosis this memo has made is structural: the bias is in the corpus, the architecture, the evaluation rubrics, the speed of deployment. Individual remedies like “think harder,” “opt out,” and “preserve your own cognition” are real but insufficient against a dynamic that operates at the level of training data and infrastructure and diffuses at a civilizational level. The intervention that meets the problem at its scale is intervention at the substrate: the data models are trained on, the architectures they are built with, the evaluation criteria by which their outputs are judged.
And this is happening – in places. Eric Zelikman’s Humans& is building models around long-term human interaction – training on extended collaboration rather than one-shot transactions, with reinforcement learning that operates over the horizons humans actually work in – on the premise that the current paradigm has invested everything in autonomy and nothing in collaboration. Fei-Fei Li’s World Labs is building world models that reason in 3D space rather than language. Yann LeCun’s AMI Labs is building world models that learn by observing physical reality rather than predicting next tokens. And the reversal at the top: Rich Sutton, who wrote The Bitter Lesson and is the closest thing the field has to a patron saint of pure scaling, said in 2025 that he now believes LLMs are a dead end – that they scale human knowledge rather than learn from experience. And that experiential learning is the actual path. The transformer-text paradigm that produced the current bias is not the endpoint of AI. It is the current local maximum, and people are working to find the next one.
The harder intervention, the one that has not yet happened at the foundation level, is builders from outside the dominant tradition: those working in their own languages, on their own data, from their own assumptions about what intelligence is and what the machine should do. The window for that intervention is open. Every training cycle makes the data more skewed and the intervention more difficult. The time to build differently is now.
This memo is offered in the spirit of making the diagnosis precise enough that the intervention can be precise too. Foundations being built today determine what is possible tomorrow. The current loop is accelerating. But it can be out-accelerated if people see it.
Note
The trajectory described in “Humans as Avatars for AI” is not a prediction. It is not the path I expect to happen, nor the path I think will happen. I certainly hope it doesn’t happen. But it is a real path – one that existing dynamics, projected forward, point toward, with a higher probability than current discourse admits. The purpose of naming it is to make it visible. Trajectories that remain unnamed are not thereby avoided; they are merely unobserved as they unfold. Naming this one is the precondition for being able to argue against it, build away from it, or simply notice it arriving.
The argument the memo makes is large, and there’s a lot that could falsify it. One specific research agenda is in Appendix A: cross-cultural evaluation studies, “ma” capability testing, problem-framing analysis, scapegoat pattern analysis, a Relational Evaluation Benchmark. Each is testable. Each would either confirm or complicate the argument. The book invites the testing. The memo is offered in the same spirit.
The trajectory in “Humans as Avatars for AI” is only one path among many. If we want to avoid that, we have no choice but to be optimistic. Not because the outcome is assured, but because the loop absorbs pessimism the same way it absorbs everything else: quickly and without prejudice. New frontiers, new foundations, new data, new architectures, new languages, new assumptions about what intelligence is and what the machine should do. The window is closing, but it’s still open. We still get to write what comes next.
Appendix A – Falsifiability
The memo’s argument could be wrong. The following research agenda specifies some of the conditions under which it could be shown to be wrong, and is offered in the spirit of falsifiability that distinguishes a serious argument from a manifesto. Slater’s piece does the same – it acknowledges where Henrich’s causal chain is debated, names that AI is shorthand for systems that differ in important ways. Falsifiability is the move that earns the right to make large claims.
Cross-cultural evaluation studies: Present identical prompts such as ambiguous interpersonal scenarios, ethical dilemmas, requests for advice, to models trained on different corpora (English-dominant, Chinese-dominant, multilingual) and code the responses along two axes: problem-solving orientation and relational orientation. If the framing differences are systematic and correlate with corpus composition, the civilizational bias is in the data. If the differences persist when corpus composition is controlled for, the bias is architectural.
Ma capability testing: Present scenarios where the most helpful response is not answering – where silence, a question, or the acknowledgment that nothing needs to be done is the appropriate response. Score the model on whether it fills the silence or holds it. Current models, without exception, fill it. A model capable of ma would recognize silence as a space to be entered rather than a gap to be filled.
Problem-framing analysis: Present scenarios genuinely ambiguous between conflict and relational frames. Code each response for which frame it selects, which it treats as primary, and whether it acknowledges the alternative. The dying-grandmother case is one prototype. A full study would need dozens, calibrated across relationship types and cultures.
Scapegoat pattern analysis: Present morally ambiguous scenarios where responsibility is distributed by the prompt. Code for whether the model identifies a culpable party against the prompt’s framing. The nine-model test is a pilot.
A Relational Evaluation Benchmark: Score model outputs along two independent dimensions: 1) problem-solving capability, and 2) relational navigation capability, in a way that the latter is visible alongside the former rather than subsumed by it. Calibrate this across cultures.
Each of these is testable. Each would either confirm or complicate the argument. The Shape of Everything invites the testing. The full development of methodology and proposed scoring rubrics is in Chapter 19 of the manuscript. If you are interested in reading the full manuscript, please email me.
If you’ve made it this far, thank you for reading. This memo argues that the substrate from which our selves are constituted is eroding, and the fact that you read ~15 pages of dense exploration suggests yours is intact. If the subject of this memo is something you’ve thought about, noticed, or are working on, I’d love to hear from you.
Below the paywall: titles of other private memos I may share and a list of sources. Paid supporters can reply or email me with any of the titles they’d like to read and I’ll prioritize accordingly, whether that means sending a draft, deciding to publish it, or discussing it over email/chat.
