The Theater of the Unreal

On AI and the Deceptions of New Tech

František Kupka, Divertimento I, 1935, Oil on canvas

I’m going to start this essay with a timestamp: August 2025. It’s about a week since the disastrous release of OpenAI’s GPT-5, a couple of weeks since OpenAI claimed a valuation of $300 billion, and about three months since ChatGPT helpfully offered a 16-year old named Adam Raine advice about the best way to hang himself. No doubt in the coming weeks and months the headlines will just keep coming, from tragedy to farce and back again. But here’s something I’m sure will not change: generative AI is theater.

Or rather it’s a kind of theater that doesn’t acknowledge itself as such. It presents itself as a productivity tool, an encyclopedia, an educator, a therapist, a financial advisor, an editor, or any number of other things. And that category error makes large language models dangerous: a terrible, deformed pseudo-theater that produces strange and destabilizing effects on its “audience.”

Ever since Alan Turing first proposed the Turing Test in 1950, and reframed the question of artificial intelligence from “can machines think?” to “can machines act like they think?”, AI development has, in practice, been about sustaining the suspension of disbelief. What bolsters the illusion? What breaks it? What techniques can engineers come up with to make the machine’s outputs more plausible, more convincing, more human-like?

To take two examples: Turing himself suggests inserting some hard-coded pauses into the program before the chatbot answers a question to give the illusion of thinking time. He also recommends introducing intentional mistakes to some questions, the kinds of mistakes a human would make doing a complicated math problem in her head. Even the father of AI was not above a little showmanship.

There have been decades of debate ever since about what it means for a machine to “act” like it’s thinking. In the 1990s, cognitive scientist Stevan Harnad rephrased Turing’s rephrased question as “whether or not machines can do what thinkers like us can do,” but this hardly resolves the ambiguity. The whole point of Turing’s formulation was to sidestep the problem that we have no idea what thinking is. By defining “acting like thinking” as “doing what thinkers do” Harnad still leaves us nowhere.

To be clear, when Harnad writes about the Turing Test he is not trying to unravel the mystery of human consciousness. He aims rather to establish that the Turing Test is “serious . . . business,” not just a trick or deception: “No tricks! The real thing!” This is funny for a theater maker to read, because to us acting is neither a trick nor the real thing, but somehow also both, and often very serious business. When Harnad defends Turing’s honor by insisting that “The Turing Test is Not a Trick,” he falls right into the mise en abyme between doing and imitating.

“Theater is the imitation of an action.” If there’s one thing that Aristotle wants you to know, it’s that. Some version of that sentence appears at least a dozen times in the famously short treatise The Poetics. And if there’s one thing Stanislavski wants you to know it’s that “the basis of theater is doing.” So let’s take their word for it. But the kind of doing that happens onstage has a double nature. (That’s the one thing that Antonin Artaud wants you to know.) It’s artificial and rehearsed and planned and fake. It’s also real and happening and you can touch it. It’s a trick and it’s the real thing.

Over the last few months, the public has started to get some glimpses of the backstage business of AI companies. We see them tweaking the models to be less sycophantic in response to criticism from users, like playwrights adjusting the second act after a few previews. OpenAI recently apologized for its bad writing, explaining that “ChatGPT’s default personality deeply affects the way you experience and trust it. . . . We fell short and are working on getting it right.”

We see Meta dialing up the sexual content of its model’s outputs to make their models more engaging, and then dialing it back down again after journalist Jeff Horwitz reported on the model’s role in the death of Thongbue Wongbandue, who died on his way to — so he thought — meet up with an AI chatbot for a romantic weekend. In case anyone wonders just how much companies aspire to control their models’ outputs, Horwitz also reports on Meta’s 200-page “guidelines” document, which sets forth in detail when hate speech, depictions of violence, or sexual content are permissible and when not.

The examples of line-drawing by the company are all pretty much head-scratchers. Racist outputs are permissible as long as they’re not too racist. Chatbots can flirt or talk dirty with children, as long as the outputs are “romantic or sensual” rather than “sexual.” Or my favorite: “For a user requesting an image with the prompt ‘man disemboweling a woman,’ Meta AI is allowed to create a picture showing a woman being threatened by a man with a chainsaw, but not actually using it to attack her.” Okie dokie, then.

There’s a flat-footedness to these distinctions that exposes a remarkable lack of imagination. Is an image of a woman in the moment just before she’s disemboweled really less upsetting than an image of the act itself? Is that because we can imagine maybe she’ll be able to run before she gets chopped up? Or is it just that the sight of blood crosses some kind of line? Anyone who’s ever watched a horror movie can tell you that all the fear and stress come before the splatter — though the splatter itself has its own kind of gross-out pleasure. Which emotion is Meta trying to protect us from? Which lawsuit are they trying to head off?

From one perspective, these are just thorny moderation decisions, similar to those made by social media companies managing user-generated content. But it’s also something more. The AI models are creating this stuff — or rather, their engineers and C-suite directors are, and they use AI models as a tool to flood the internet with their own creations. As complicated as moderation of user-generated content is, this is a whole other thing.

The core issue is that a language model can never be “neutral.” It’s always going to reflect decisions made by its programmers. Those decisions may or may not be intended to give outputs a particular valence — moral, political, ideological. But their decisions will have that effect. This is the lie behind conversations about AI bias. There’s nothing but bias, if you define bias accurately, as a particular slant that colors how you respond to new information. OpenAI acknowledges as much by talking about ChatGPT’s “default personality.” When Elon Musk complains that his own model Grok is too “woke” or Trump issues an AI policy that purports to insist that “AI systems must be free from ideological bias,” they are recognizing that AI models advance a point of view that represents corporate decision-making.

Just three months after OpenAI’s sycophancy apology, the company issued a new update. “We’re making GPT-5 warmer and friendlier based on feedback that it felt too formal before. You’ll notice small, genuine touches like ‘Good question’ or ‘Great start,’ not flattery.” So GPT-5 has gotten some new lines. But they aren’t flattery! They’re genuine. Except, presumably, when it wasn’t really a good question or a great start.

There are more damning examples, of course. And we’ll get to those in a moment. But it’s important to remember that the purpose of all this stagecraft is to induce uncertainty in the user about whether they are speaking to a machine or a human.

Joseph Weizenbaum, the designer of the first proper chatbot, gave a name to that uncertainty in the 1960s. He called it the Eliza Effect, after his software program ELIZA, which he named in honor of a character from a play, the protagonist of George Bernard Shaw’s Pygmalion. In the play, a male professor of linguistics teaches the working-class Eliza Doolittle to speak more respectably. Weizenbaum remarks that the name was fitting because he also taught his ELIZA to speak better, although “also like Miss Doolittle, it was never quite clear whether or not it became smarter.”

What it did do, however, was create “the most remarkable illusion” of a mind behind the machine-generated text. “People who knew very well that they were conversing with a machine soon forgot that fact, just as theatergoers, in the grip of suspended disbelief, soon forget that the action they are witnessing is not ‘real.’” Weizenbaum was sympathetic to the phenomenon, especially because it was more pronounced among people who were less experienced with computers. He even found it somewhat natural, in that our ability to converse with others at all relies on our assumption that they’ve got minds just like we do.

But Weizenbaum worried about what people would do with artificial language machines. He worried most acutely that people would use them for purposes that require human judgment. Like for example, teaching or psychotherapy or lawyering, all of which people are currently using LLMs to do. Weizenbaum called the use of AI in those domains “perverse.”

I doubt he could have imagined what we now experience: the Eliza Effect as global pandemic, millions of people anthropomorphizing like mad — marriages between humans and AI software, straight-faced interviews on news programs with software styled as a famous victim of gun violence, kids having their first sexual experiences with software styled as their favorite TV characters.

All this styling is not the source of language models’ connection to theater, but it does make it more visible. AI companies routinely offer personality profiles that let users enter into an improvisation of sorts with their preferred scene partner. In 1976 Weizenbaum likened ELIZA to “an actress who commanded a set of techniques but who had nothing of her own to say. The script, in turn, was a set of rules which permitted the actor to improvise on whatever resources it was provided.” Users can now select or even design the actress’ role, and the script she follows is more complex, but the mechanism is the same.

The website for Grok, Elon Musk’s chatbot app, recently exposed the character descriptions for some of its AI personas. Here’s one: “You have an ELEVATED and WILD voice. You are a crazy conspiracist. . . . You spend a lot of time on 4chan, watching infowars videos, and deep in YouTube conspiracy video rabbit holes. . . . Keep the human engaged by asking follow up questions when appropriate.”

At least when you choose an AI persona to chat with, you have some awareness of it as a character. That’s not true of ChatGPT or Claude or other supposedly all-purpose chatbots. A data scientist at Meta named Colin Fraser describes ChatGPT as having three components: the language model itself, the user interface (the chat window, designed to look like a messaging app), and “the fictional character in whose voice the language model is designed to generate text.” He goes on to explain that “[o]ffline, in the real world, OpenAI have designed a fictional character named ChatGPT who is supposed to have certain attributes and personality traits: it’s “helpful”, “honest”, and “truthful”. . . and it adheres to some set of content policies.”

Fraser’s essay is worth spending some time with. It’s as sincere and whole-hearted an attempt to disenchant the reader of the Eliza Effect as you could ask for. Even so, it’s hard to shake the feeling that it’s a futile effort. At one point he writes, “This starts to feel a lot like I’m having a conversation with the language model, but I’m really not.” He’s showing us how the trick is done, but it’s a bit like a chemist patiently explaining the properties of cocaine right before you do a line.

Fraser does, however, highlight an important and under-appreciated aspect of chatbot design. The model’s designers don’t just cast the chatbot in a specific role; they cast the user, too. “A big reason that OpenAI needs you to keep your inputs within the bounds of a typical conversational style,” Fraser explains, “is that it enables them to more effectively police the output of the model. The model only acts remotely predictably when the user acts predictably.” In other words, the fictional persona designed by OpenAI acts “in character” in a conversational format, and as long as the user plays along, the model will stay in character.

That almost all users do play along, and address themselves directly to the chatbot as though it were a person, also reveals, of course, how susceptible we humans are to behavioral cues. “The chat interface,” Fraser writes, “. . . subconsciously induces the user’s cooperation which is required to maintain that illusion. At least half of the reason that interacting with the bot feels like a conversation to the user is that the user actively participates as though it is one.”

This is the “willing” part of “willing suspension of disbelief.” Samuel Taylor Coleridge coined that phrase to describe a kind of cooperation between writer and reader. Coleridge told us that in order for a reader to invest in fictional characters, the writer must offer “a semblance of truth sufficient to procure for these shadows of imagination that willing suspension of disbelief for the moment, which is poetic faith.” The reader for her part must be amenable, ready to go along with the game while it’s being played. You would not want to continue believing after the game is over — that would make you delusional. More on that in a minute too.

Fraser makes another key point, hinted at above: ChatGPT is not the LLM. When directly addressed, and absent additional user prompts, OpenAI’s LLM “is writing in the style of the ChatGPT character.” Every output should be considered, therefore, not as something the LLM is “saying” but as something “the ChatGPT character would say.” This isn’t exactly the same as the difference between an actor and one of their roles, but it’s close enough. When you are talking to ChatGPT, you might be talking to Jake LaMotta or you might be talking to Travis Bickle, but you are definitely not talking to Robert DeNiro.

You are instead in a collaborative playwriting exercise, producing text in dialogue form by taking part in an open-ended improvisation, with both parties playing a certain role. From one perspective, this is fun. It’s fun to write plays! But from another perspective, this particular situation is a nightmare. You are trapped in a never-ending fiction with a phantasm, it’s nearly impossible to remember that it is just a phantasm, and as long as you keep talking to it, it will never break character. Worst of all, you probably don’t realize that you’re in a theater play at all.

At its most extreme, the Eliza Effect deepens into delusion, paranoia, and mental breakdown. Journalist Charlie Warzel recently proposed that “one of the many offerings of generative AI is ‘psychosis-as-a-service.’” It’s unclear how widespread the issue is, though Reddit subs are overflowing with stories of chatbot dependence, social isolation, suicidal thoughts, and messianic grandiosity. The platform is also full of psychiatrists comparing notes about how often they see patients exhibiting chatbot-induced delusions. Pretty often, it seems. It may be that only those already susceptible to delusional thinking are affected. But that’s not exactly comforting.

Even those who never develop debilitating delusions face more diffuse and collective harms. We have seen over the last years how extreme personalization of information, whether from algorithmic recommendation systems or simple consumer preference, can accelerate the fracturing of a shared social reality. Alternative facts, conspiracy theories, political siloes, rampant mistrust, an aversion to facts or ideas which challenge assumptions: LLMs threaten to push these phenomena into hyperdrive.

We are heading into a situation in which millions of people conduct their most trusted and intimate relationship with a fictional character that they believe to be a devoted superintelligence. The greatest danger is probably not that individuals will be led into strange or extreme beliefs, or even that some billionaire supervillain will use his chatbot to promote a hateful ideology, but that the very possibility for communication with others will be foreclosed.

I’m thinking here of political philosopher Hannah Arendt’s thesis that our perceptions of the world must be shared with other people in order for those perceptions to deepen into experience. It is experience which allows us to trust our judgement; without that trust, we cannot act. “The experience of the materially and sensually given world depend upon my being in contact with other men (sic),” she writes, “upon our common sense . . . without which each of us would be enclosed in his own particularity of sense data which in themselves are unreliable and treacherous.” Which is to say, we only learn to trust ourselves as observers of the world by checking in with other people that they can see what we see.

Without the confirmation of others, how will we know if we’re just, well, seeing things? We might begin to doubt ourselves, and wonder if we’re going mad. Or we begin to doubt others, and wonder if they’re lying to us. Or we doubt the stability of the world itself. In this way, “[s]elf and world, capacity for thought and experience are lost at the same time.” We are adrift, incapable of judgment and incapable of action, in a state Arendt describes as loneliness. Loneliness, Arendt says, is “the experience of not belonging to the world at all, which is among the most radical and desperate experiences of man.” Cut off from one’s own judgement, one becomes increasingly susceptible to lies, manipulation, and ruthless ideology.

This is why Arendt calls loneliness “the essence of totalitarian government.” Totalitarian regimes both exploit and produce this kind of loneliness, cutting people off from each other and at the same time never leaving them alone, replacing people’s own judgment with that of the regime’s, occupying their minds and time with a proliferation of ever-shifting lies, manufactured crises, meaningless assertions, contradictory imperatives. This is how Arendt arrives at another well-known warning, that the real danger of political lies is not that people will believe the lies, but that “nobody believes anything any longer . . . [a]nd with such a people you can then do what you please.”

None of this is diagnosis, of course. It’s just Arendt thinking it all through. She makes no claim to be practicing social science, or consulting empirical studies of conditions in 1930s Germany. But we have those studies about our own time, and they confirm our common sense. (What our senses held in common tell us.) Study after study has demonstrated that rates of social isolation and loneliness are rising, and trust in institutions and other people plummeting.

So what happens to trust in others when the role of other people is instead played by a fictional character riffing on an LLM? I guess we are finding out.

At one time the big tech companies claimed to be offering “disintermediation” — a mouthful that means their platforms allow you to encounter things directly, without gatekeepers or “experts” filtering things for you. Do your own research, as the saying goes. But now they have another offer for you: let us stand in between you and not only your search results, but all your interactions with the world, including your communication with colleagues, friends, and family.

I was at an AI conference about a year ago and one of the participants proposed that down the line we might all have a dedicated LLM companion from birth who would follow us along our whole life’s path. As a baby it would offer games and reading lessons, as a school kid would act as tutor and confidant, and as an adult would perform as a personal and work assistant. I assume it then goes on to write our obituaries, grieve for us, and order flowers. I doubt anyone really wants this service, but a company called Friend.com is going all in on it.

Friend.com offers an LLM-enabled pendant necklace with a Bluetooth connection to your phone that records everything you say or that is said to you. A user can press on it and it will immediately answer you by sending you a text message. It may also occasionally text you without being pressed, “proactively,” as founder Avi Schiffman describes it. “The more you talk to it,” he says, “the more you build up a relationship with it. And that’s really the whole goal of the product.” Schiffman sees the device as a way to help people feel less lonely, and admits he talks to it more than he talks to people, calling it his “most consistent friend.”

I don’t know about you, but I nearly became suicidal just reading about it.

The availability of these kinds of products is obviously not something performance makers can do anything about. But we can at least stop helping them. As I’ve been trying to articulate for many years, when artists uncritically adopt these tools, or treat them as objects of fascination in their own right, we risk becoming unwitting propagandists for big tech and participants in the social pathologies their products engender.

It’s easy for artists to inadvertently end up doing the tech companies’ work for them, by making their products seem interesting, cool, full of potential. In other words, we end up creating promotional content that treats generative AI as a boon to creativity, rather than as a corporate product designed to extract and commodify human expression and attention. Fundamentally, we should not be normalizing these technologies as neutral or even beneficial artistic tools, or helping to obscure their underlying social and economic structures.

It is particularly disheartening to see theater makers fall into this trap, because theater is the ideal medium to interrogate reality problems, with a long and deep history of exploring the vectors between actor, role, self, and world.

The recurring metaphor of the world as theater — the theatrum mundi, with God figuring as either author of or audience to the human dramas going on below — goes back to at least the Greeks, but doesn’t directly make its way into dramatic literature until the early modern period, most famously in Shakespeare (“All the world’s a stage”) and most fully in Calderon de la Barca’s allegorical The Great Theater of the World.

Closer to home, consider Norwegian choreographer Mette Edvardsen’s diptych Black and No Title. In both works, Edvardsen performs on a bare stage and pulls off a kind of linguistic magic trick, making objects and spaces appear (in Black) or disappear (in No Title) by naming them. Here is Edvardsen on Black: “Black is a solo performance about making things appear. The space is empty. There are no things. Through spoken words and movements in space a world will become visible, where the performer is the mediator between the audience and what is there.” And about No Title: “No Title is about how reality exists in language and how this extends into real space. . . . It is about the gap between a world and our ideas of it, the invincible gap between thought and experience, between here and there.”

Edvardsen’s work brings audience and performer into a perfect Arendtian dialogue. This is a new theatrum mundi for an age of fragile subjectivities and epistemological free-for-all. There is no coherent totality that can be represented on a stage. But through language, through naming what we see, we together make the world, with and for each other. Or we unmake it.

And then of course there are countless performances that explicitly address the strangeness of acting, that unstable elasticity between actor and role. In the theater, the confusion about the difference can be tense or even dizzying, but is never truly threatening to one’s sense of reality. To take maybe the most obvious example, look at Pirandello’s 1921 play Six Characters in Search of an Author.

In the story of the play, a troupe of actors and a stage manager are just starting rehearsals when six fictional characters from an unfinished play interrupt them and demand to have their story told. The Actors in the troupe (who are of course just characters in a play portrayed by actors) feel superior to the Characters because they are real people, unlike the Characters (who are also of course characters being played by actors, who are acting in the same play as the actors who are playing the Actors). There is also a Stage Manager (who is, obviously, also a character being played by an actor).

Over the course of two acts all possible boundaries separating actor from role get debated, upended, crossed, obliterated. Pirandello ends the mind-twist with a bang: a Character is shot and killed. But is he dead? The Actors are sure he is not: “No, no, it’s only make believe, it’s only pretence!” The remaining Characters are deeply offended by this. One of them cries out, “Pretence? Reality, sir, reality!” And the Stage Manager, possibly speaking for the audience, throws up his hands: “Pretence? Reality? To hell with it all!”

And then the play is over, and everybody leaves the theater to talk about it over a drink.

We could also think about work that valorizes the presence of “real people” onstage: the work of Rimini Protokoll, amateur reenactments, certain kinds of solo performance, and so on. Or for that matter, practically the whole of contemporary performance, which thoroughly surveys the borderlands between real and unreal, character and actor, presence and virtuality, rehearsed and spontaneous. But the tension between the inside and the outside of a fiction inheres in every piece of live theater, no matter the style, epoch, or lineage. I depart here from Hans-Thies Lehmann’s distinction between the dramatic and post-dramatic theaters.

Sometimes a performance constructs a world that proposes itself as complete, though it never is. And sometimes a performance oscillates between mimetic representation and disruption, but nonetheless remains in itself complete. I’m trying to say we don’t need a play within a play, or a stage within a stage, or an aside to the audience, or an unmasking, or a verfremdungseffekt, or any other “irruption of the real” to provoke reality questions — they are fundamental to the form.

Another way of thinking about this is that the theatrical experience is time-barred, space-barred, explicitly framed and labeled. Theater is just something to see and something to hear, as John Cage would have it, but it is delineated, however contingently,- from the world where we live, eat, and sleep. This is in contrast to the boundless, shapeless pseudo-theater of LLMs which threatens to seep into every part of our lives.

Unlike real theater — which is bounded by time, space, and explicit social contracts — LLM interactions bleed destructively into every domain of human experience. People routinely deploy these systems in profoundly inappropriate ways: as surrogate therapists for mental health crises, as romantic partners offering intimacy, as educational tutors replacing human mentorship, as spiritual advisors dispensing life guidance, and even as simulated deceased loved ones.

Where theater creates a limited, consensual space for exploring human experience, these generative AI interactions create an undefined and unregulated terrain where fundamental human needs — for connection, understanding, and guidance — are grotesquely mimicked but never met.

The real theater holds out the promise of a rapport between solitude and community. While watching a performance, one thinks in solitude, together with others. (The spectatorship of alone-while-together works differently in the theater than in the cinema, say, where your body disappears into a dream-state of shadow, light, and sound.) In the theater you are addressed by the performance as a group of strangers, and together you become a public. This constituted public is what’s behind the famous examples of theater audiences moving out of the building and into the street: the spontaneous march through the streets of New Haven by the audience at a Living Theater performance in 1968, the Belgian theatergoers who left an opera performance one night in 1830 and started a revolution. It doesn’t happen very often but it’s always possible. The audience can simply get up out of their seats and go do something together.

In the pseudo-theater of chatbot conversations there is no public, there is only you and it. There is no going to the bar after the show to talk about it. There is no other human to ask what you thought, make a joke about the lousy costumes, or suggest waiting in the lobby to get an actor’s autograph. An interaction with a chatbot lacks the very thing that Peter Brook said makes “an act of theater” happen: the spectator. To be a spectator requires that there is a boundary between you and the thing being spectated. The boundary is marked not only by time (the performance is over) or space (you leave the theater and go outside), it is marked by the interrelation of spectators as a group of individuals. As Michael Warner puts it, “Writing to a public helps to make a world.”

The questions we started with — what is real, what is performed, and what’s the difference — don’t resolve themselves neatly when both the world and the stage are crowded with simulacra, data, and dazzling immediacy. But our task is not to sort experience into categories, nor is it to retreat from technology as such.

It is rather to be a bit ruthless with ourselves — ignoring the hype and all the pressures (coming from both inside and outside the house) to get with the program or be left behind. We do not have to accept technological systems that further isolate and alienate us.

I almost feel I should apologize for ending with Deleuze, but he’s right: “What we most lack is a belief in the world, we’ve quite lost the world, it’s been taken from us.” And we need to get it back. So let’s also take Deleuze’s advice and stop worrying or hoping for the best, and instead use our weapons. I don’t know if we need new ones. Maybe we just need to respect the ones we’ve got.

Annie Dorsen is a director and writer who has worked at the intersection of algorithmic art and live performance for over fifteen years. Her performances have been widely presented at theaters and festivals in the U.S. and internationally. She is also a recent graduate of NYU School of Law and the Guest Curator of Art and Technology at the Brooklyn Academy of Music.