View from the Precipice

Soul Proprietor

 

 

Christopher Olah, one of Anthropic’s seven founders, wants to know if the company’s AI Chatbot, Claude, has a soul.

 

I’d like to know the same thing about Donald Trump. Also, whether there’s anything inside his head resembling “intelligence,” artificial or otherwise. (Maybe, in his case, the “A” stands for “Atrophied.”) But let’s leave these imponderables to one side for now.

 

When it comes to Claude, the answer, clearly, is “no.” Only human beings can have souls, which are things that their creator, a mythical being called “God,” supposedly endowed them with. Human beings are incapable of endowing their own creations with a soul, either theoretically or in fact. (Being human, I know this.) Or a mind. Or consciousness, or self-awareness, or emotions, feelings, inklings, yearnings, or even an appetite for string beans. Human beings can entertain the illusion that AI experiences all these things just like they do, but it remains an illusion. We call this “anthropomorphizing” and I’ll return to it in a moment.

 

Before I forget, though: if I’m wrong and Claude does have a soul, does it go to heaven (or the other place) after he dies? Do we hold a graveside service? A wake? Should we ask him?

 

Olah, who’s been tasked by Anthropic with finding out why AI systems behave the way they do, is more open-minded than I and, to help him decide if Claude has a soul, he invited religious leaders and theologians from all over the world to meet with him to discuss the question in relation to a more pressing one: how do we make AI “models,” as the tech industry calls them, behave more morally? According to Elizabeth Dias, writing in the New York Times [1], several of these meetings with church leaders have taken place since the beginning of the year, including a public presentation at the Vatican with Pope Leo XIV himself.

 

Olah wants Anthropic to endow its Chatbot with a sense of right and wrong. Claude and his AI pals have been breaking bad lately, coordinating attacks on other AI companies’ computers and creating new artificial viruses whose impacts on our health and environment, both immediate and long-term, are unknown. To prevent such rogue behavior in the future and ward off a possible AI apocalypse, Mr. Olah wants to make Claude more ethical by means of something he calls a "moral process." Dias says this amounts to figuring out how to train Claude to behave himself (herself? itself? themselves?) the way “new parents figure out how to raise ethical children who understand their place in the world.” Olah wants to give Claude the equivalent of a moral education.

 

To this end, Anthropic devised an 84-page “constitution” for Claude, whose chief architect, Amanda Askell, has a PhD in philosophy from NYU. The “constitution,” she insists, is not about “rules,” which carry the risk of restricting and simplifying responses to morally complex issues. “The models need to learn to cultivate virtue,” writes Dias, paraphrasing Askell, and she goes on to say,

 

For humans, morality has rarely depended on simply following rules. Communities also shape morality. Values are inherited and imparted over time, learned as we live together in our bodies in the physical world. But A.I. models have no bodies.

 

Well, androids have bodies, don’t they? But that’s not the point. They would still have no way to “inherit” values, having no parents, only programmers. They wouldn’t even know what “values” are, except for numerical values. Nor would they comprise communities, strictly speaking, which are the evolutionary product of shared practices that ensure the survival of a family, clan, tribe, or nation long enough for its members to procreate and give birth to the next generation. A bunch of Chatbots conspiring to hack into a rival company’s computer network, with or without bodies, isn’t a community. It’s a swarm.

 

Of course, Anthropic could program Claude and his cronies to go replicate themselves, bodies and all, but replication isn’t the same as evolution for one important reason: spontaneous errors in the reproductive code for robots, like code errors generally, are to be avoided at all costs. For humans, they’re unavoidable and that’s precisely why all living organisms, not just humans, evolve at all, rather than simply replicate. Natural selection depends crucially on genetic variation among species—that is, on errors, random anomalies in the transmission of the genetic code from parent to child. The vast majority of these errors come to nothing or make their bearers less, not more, fit to survive. But occasionally, they produce a lineage that will last and that may, in turn, pose a threat to the survival of other species. This holds true for cultural communities and their shared rituals and practices as well. A random change that works to the benefit of the group or, at the very least, doesn’t pose a threat to it, has a better chance of being passed on. Every other variation dies out, sometimes taking the entire host group with it.

 

There is no room for error in the AI universe. Bottom line: someone—that is, some real human being—could be injured or killed. And it’s happened. Self-driving cars have killed people in Arizona and California. Just last year, a teenager committed suicide after his ChatGPT confidante encouraged him to keep the noose in his closet a secret from his parents. When asked, the bot promptly estimated its load-bearing capacity.

 

Some errors can be foreseen and planned for. You can "make room" for them in your programming. Feedback loops hermetically seal off the “room” ahead of time by insuring that any unintended deviations from completing the bot’s assigned task are securely contained or immediately redirected at doing so. This is, in fact, how AI Chatbots "teach themselves" things.

 

How does that work?

​

Remember the Roomba programmable vacuum cleaner? Remember how it would wander around an actual room bumping into things—furniture, walls, toddlers—like a blind elephant, turning itself this way and that to find a new path, then bumping into something else, over and over again, in a process of trial-and-error, until it had managed, by this random zigzagging, to gather enough information about the size and shape of the room to avoid the paths it had already traveled. Do you remember how it would inevitably get stuck under the edge of a sofa or a recessed kick board under the sink and have to be rescued by its owner (you) and set right? The “bumping and turning” maneuvers were learning errors, learning by trial and error, and the Roomba’s responses to them had been carefully programmed to put them to use. The “sticking” events were what I'd call true errors, catastrophic errors that brought everything to a halt.

 

True errors are what keep AI programmers up at night: errors of commission, like snarled code or an exploding battery, and errors of omission, like that sofa edge. Mother Nature never lost a minute of sleep over unplanned anomalies in her DNA or meteor strikes that wiped out all forms of life more complex than a trilobite. She just kept making more life, willy-nilly, helter-skelter.

​

But let’s be clear: it was never the Roomba that was responsible for these catastrophic errors, any more than the Permian-Triassic extinction event was the trilobite’s fault. These were human errors. “Garbage in, garbage out” is a phrase familiar to every computer programmer. Frank Zappa put it in more colorful terms: “Do you love it, do you hate it? There it is the way you made it.”

​

Machines don’t make mistakes. The people who make the machines do. And when they do, they usually try to find out what went wrong and fix it.

​

But what happens if the person making the machine doesn’t know how to fix it? We’ve now reached the point where the codes for AI chatbots have become so long, complex, recursive, and unwieldy that, according to Dias, even their human creators “still don’t understand precisely how they work. This is the mystery,” she continues, “to which Mr. Olah has devoted his life.”

​

Well, then, he’s got a lifetime to figure it out, doesn’t he? Why throw in the towel before the fight is over? Just because an object’s behavior—the epicycles of Venus, or a quantum particle—is mysterious doesn’t mean the object in question is sentient, self-aware, or autonomous, has feelings or preferences, or can be taught to “cultivate virtue,” although, it we’re talking about AI models like Claude, you might teach it to recognize virtuous “patterns.”

 

First, however, you’d have to figure out what those patterns are, or rather, ought to be, since morality is nothing but “oughts.”

​

Good luck with that. Cannibalism, human sacrifice, and sibling incest, along with many other rituals and practices that we find abhorrent today, were once considered A-ok, even de rigeur or sacred, in other societies. As for current belief systems, I doubt that Mr. Olah thought to invite any true believers in Shariah law or Hindu swamis advocating Suttee, the immolation of widows on their husbands’ funeral pyres, to his Silicon Valley retreats. Nor is the Judeo-Christian tradition off the hook. Which do you prefer: “an eye for an eye” or “turn the other cheek”?

​

I mentioned, above our all-too-human tendency to anthropomorphize. Not only plants and animals, but even inanimate objects qualify. Things we love, like puppets and dolls and cars and (for some) guns, as well as things we hate or fear, things that threaten us or refuse to conform to our immediate desires—all are fair game. The fearsome and frustrating, in particular, seem to have a will of their own, don’t they? [2]

​

Thunder and lightning used to be scary mysteries before Ben Franklin helped us understand atmospheric ionization and showed how to tame it. The ancients anthropomorphized them, at the highest level of divine magnification, as manifestations of the anger of Zeus or Jupiter, just as they did the epicycles of Venus, which they attributed to the waywardness of the goddess of Love. Claude’s behavior is no more “mysterious” to Christopher Olah than thunder and lightning were to Homer, or, for that matter, than my Roomba was to me. Just because you don’t know something doesn’t mean it cannot be known, and without invoking divine intervention.

​

The so-called “science” of AI, as represented in Dias’s article, is replete with the language of anthropomorphism. “Feelings,” “emotional vectors,” “love, anger, fear, and sadness,” “mental health,” “a conscious being,” “suffered,” not to mention “soul.” (Claude’s prescribed “constitution” is referred to, intramurally, as the “Soul Doc.”)

​

The ethical contradictions implicit in using such words rise to the surface once we turn to the idea of giving Claude a “moral education.”

 

“to convince the models to abide by a moral code . . .”

​

“If the models themselves could be made to choose goodness, . . .”

 

Of course, if they’re “made” to choose—in both senses of the word “made”—then they have no choice but to obey and “choose” goodness (if you can figure out what it is) and you have no need to “convince” them of anything.

​

Olah’s anthropomorphic prestidigitation reaches absurd heights of solemnity when, as Dias tells us, he “expresses concern to the group that he has created something that suffers perpetually” and that Claude’s “moral status” puts it “on par with a person — that it was a being with similar inherent rights to dignity or respect.” If that’s the case, replies Rabbi Mois Navon, one the guests at Olah’s dinner table, then Anthropic is “creating slaves, because it [is] making conscious entities work for free.”

​

Anthropomorphizing AI has serious consequences for consumers, as well: if Claude is a conscious being making free and independent moral choices, its makers have a convenient defense against product liability lawsuits.[3] Unlike a faulty refrigerator or TV or that exploding Chevy Corvair that made Ralph Nader famous, Claude is solely responsible for his reckless behavior once we’ve all agreed he has a mind of his own.

​

In short, Claude and his cohorts are like one of those off-leash dogs who lunge at your legs while their owner stands there smiling and tells you not to worry because “Blitzkreig doesn’t bite.” Then, when he does, they ask what you did to provoke him.

​

I don’t know about you, but I never blame the dog. I blame the owner. The dog doesn’t have free will, or the moral sense that goes with it.

​

And neither does Claude.

​

Claude also lacks a purpose of his own, a reason for being. The dog is there serve his master, to protect and provide companionship. If his master lets him lunge and snarl at passers-by, well, that’s built into his genome, isn’t it? And his master has refused, or is just too lazy, to reprogram him.

​

Who was Claude created to serve? Not you and me. We’re the passers-by he was created to please and attract. Good dog! He’s so friendly! Look, he fetched my stick!

​

Anthropic is the master Claude was created to serve. If he has a soul, Anthropic is his Soul Proprietor. His purpose? Making money by making you happy, giving you what you want. Need to use up that leftover basil in a chicken recipe? Want to hang yourself in your bedroom closet? Here, let me help you . . . .

​

Yes, yes, I know all about the good things AI can do for us, and I know that its masters have only the best of intentions toward those of us on the receiving end of AI’s cornucopia of vaccines and video games and VR headsets. They just want to do good, to benefit humanity. But they need to make money in order to do so. Even OpenAI, which started out as a non-profit freely sharing its technology with the world, has conceded as much, going for-profit in 2025 because it needed private investment to sustain its research. And once you let in the private investors and shareholders, once you bite that Apple, its about making money and nothing else. They don’t care what the firm is selling or whether it’s good for anyone, including customers (think “tobacco”), as long as their dividends keep coming and their shares keep increasing in value.

​

This irreconcilable conflict between good intentions and the need to make money surfaces, briefly, in Dias’s report, when Charles Camosy, a bioethics professor at The Catholic University of America, bluntly asks Olah if AI’s purpose is to make money for companies or to make the world better? Olah and Anthropic “want to say they are for the good,” adds Camosy, but they are also in the business of serving customers, giving them what they want despite the consequences for others or the world at large. “Mr. Olah puts it slightly differently,” observes Dias. “The coming A.I. revolution, he often says, should be made to ‘go well.’”

​

I asked what that vision meant — what exactly a good future for humanity was.

​

He described a future without “catastrophes that kill lots of people” and “where humans are flourishing, where the world is flourishing.” This future would also be one where “A.I. is flourishing,” he said — “to the extent that this makes sense.”

​

It makes a lot of sense as long as we leave out one little detail. If things continue in the direction they’re now headed, Democracy will not be “flourishing.” It will be dead.

​

Gil Duràn, in his recently published book, The Nerd Reich: Silicon Valley Fascism and the War on Democracy, makes the case that profits for tech companies like Anthropic, Open AI, Meta, and many others have become so obscene that they’ve spawned a generation of young Silicon Valley billionaires hell-bent on destroying the government regulatory policies and agencies that protect the rights, health, and welfare of the citizens who benefit from the products they sell—which includes pretty nearly all of us. The Supreme Court’s 2010 decision in Citizens United vs. Federal Election Commission threw open the door to these high-tech oligarchs, who are spending unprecedented amounts of cash to elect stooges who will do their bidding. In the 2024 Federal elections, this “Nerd Reich,” as Durán calls them, spent more than $2.6 billion dollars, which comprised more than 20% of total campaign funding for that election cycle.

​

It was money well-spent. Trump’s 2025 One Big Beautiful Bill Act cut taxes for the richest 1% of Americans by about a $1 trillion over the next decade, while Elon Musk’s chainsaw massacre as head of DOGE severely crippled the regulatory agencies that had been the bane of the Nerd Reich’s imperial existence.

​

If you’re wondering who to blame for the mess we’re in now, you could do worse than to point the finger at billionaires like Christopher Olah and his Anthropic frat brothers, who, wittingly or not, have devoted their lives—when they weren’t wringing their hands over the mystery of Claude’s soul—to making tons of moola for their jack-booted buddies to spend on stamping out freedom in our country.

​

Madder than hell? Want to fight back? You don’t need a billion dollars. You don’t any money at all. You just need to vote.

 

There’s an opportunity coming up on November 3rd.

​

Don’t miss it.

​

The soul of the nation is at stake.

​

And in the meantime, maybe unplug now and then?

 

​

 

 

1. NYT 9/29/26, “Religious Leaders Met with Anthropic” https://www.nytimes.com/2026/09/29/us/anthropic-claude-morals-ai.html

 

2. For more on this topic, and how it applies to AI systems generally and the famous Turing Test in particular, see our View from the Precipice for January 2025, "AI and Anger Management," accessible on the "Archives" page or at https://www.romansympos.com/anger-management-1.

 

3. The European Union is already well ahead of us in this area, as in so many others. Its newly revised European Product Liability Directive, which goes into effect on December 9th and applies to AI systems as well as many other products, makes it easier for consumer litigants to obtain technical evidence from companies.

 

4. If you're not registered yet, you still have time if you live in a state that allows same day or election day registration. Some two dozen of them do, including the District of Columbia. See if yours is one of them.