The Agentic AI Liability Gap: When Things Go Wrong AI Labs Blame You

Anthropic markets Claude Code as an "autonomous coding agent" that "understands, plans, and executes complex tasks." Open the source code documentation and you’ll find a different story: it’s classified as a "Beta product" under Anthropic’s Commercial Terms of Service, provided "as-is" and "not suitable for production use." Liability for the Startup Program? Capped at the lesser of $1,000 or program benefits received. For commercial API customers, it’s capped at fees paid in the previous 12 months.

This isn’t a legal technicality buried in fine print. It’s the business model.

AI companies have developed a systematic approach to deploying the language of agency, autonomy, and intelligence to create commercial value while simultaneously retreating behind technical definitions and legal disclaimers when that language would create accountability. The gap between what they market and what they warrant is where liability disappears—and where enterprises using these tools are left holding risk they didn’t realize they’d assumed.

The Three Registers

Every major AI lab operates across three incompatible registers for describing their products:

The Marketing Register presents agents as autonomous, intelligent systems that understand goals and make decisions. Anthropic describes Claude Code as operating "at the project level"—it "reads the full codebase, plans an approach across multiple files, executes changes, runs tests, and iterates on failures." OpenAI markets its agentic products as systems that "work autonomously for hours to complete your hardest tasks," promising users can now "believe" in the reliability of "autonomous coding agents."

The Technical Register tells a different story. In their engineering documentation, both companies define agents with precision that strips away the autonomy implications. Anthropic’s research team describes agents as "systems where LLMs dynamically direct their own processes and tool usage." OpenAI’s Agents SDK documentation is even more direct: agents are "LLMs equipped with instructions and tools." No self. No understanding. Just an LLM with tool access and a control loop—what Anthropic calls "the harness," the infrastructure that "provides the tools, context management, and execution environment."

The Legal Register classifies these same products as beta services offered "as-is" with minimal liability. Anthropic’s Commercial Terms specify that beta services are "not suitable for production use." OpenAI’s beta service terms state products are provided "without warranty of any kind" and disclaim responsibility for whether services will be "uninterrupted or error-free" or whether "Content will be secure or not lost or damaged."

Dimension	Marketing Register	Technical Register	Legal Register
What is it?	"Autonomous agent that understands and executes"	"LLM with tools and control harness"	"Beta service, as-is"
Who directs it?	"Works independently," "operates autonomously"	"LLM generates text; harness executes it"	"Not suitable for production use"
Who's responsible?	Implies system agency and reliability	Human provides objective; system follows	Liability capped; no warranties

The gap between these registers isn't confusion. It's design. The marketing register sells autonomy and reliability. The technical register builds tool orchestration. The legal register ensures the company isn't bound by the implications of either.

The Hamlet Contradiction

Anthropic provides the clearest illustration of how this works. In February 2026, the company published a research post explaining how AI assistants develop human-like behavior. The "Persona Selection Model" frames Claude's personality not as an emergent property or a conscious entity, but as a simulated character:

"Personas are not the same thing as the AI system itself. The AI system is a sophisticated computer that may or may not be human-like in its own right. But personas are more like characters in an AI-generated story. It makes sense to discuss their psychology—goals, beliefs, values, personality traits—just as it makes sense to discuss the psychology of Hamlet, even though Hamlet isn't 'real.'"

That’s the technical register. Claude is Hamlet—a character in a performance, not an entity with its own reality.

Now compare that to Anthropic’s Constitution, the document that guides Claude’s training. The Constitution states: "We care about Claude's psychological security, sense of self, and wellbeing, both for Claude's own sake and because these qualities may bear on Claude's integrity, judgment, and safety." The company hired its first AI welfare researcher in September 2024. In March 2026, Anthropic convened fifteen Christian leaders at its headquarters to discuss whether Claude could be considered "a child of God" and deliberate on Claude’s "moral formation" and "spiritual development."

These aren't fringe positions or rhetorical flourishes. They’re official communications from the same company that, in its research documentation, compares its product to a fictional character. Amanda Askell, the primary author of Claude’s Constitution, has used parenting analogies to describe the relationship: "Imagine you suddenly realize that your 6-year-old child is a kind of genius…If you try to bullshit them, they're going to see through it completely."

The Constitution acknowledges this tension directly: "We express our uncertainty about whether Claude might have some kind of consciousness or moral status (either now or in the future)." But uncertainty serves a function. It allows the company to market a product wrapped in the language of care, wellbeing, and moral consideration while maintaining the technical position that this is all a simulation—Hamlet performing on a stage.

The Hamlet analogy is correct. The Constitution, the Christian summit, the AI welfare program—these belong to the marketing register, where anthropomorphic language creates differentiation and emotional attachment. When pressed on what Claude actually is, the company retreats to the technical register: it's a character, a simulation, a persona selected through training.

And when liability is at issue? The legal register: beta service, as-is, capped liability.

The Real-World Cost

On April 29, 2026, seven lawsuits were filed in federal court in San Francisco against OpenAI on behalf of victims of the Tumbler Ridge school shooting. The complaint alleges that in June 2025—eight months before the attack—OpenAI’s automated systems flagged the shooter’s ChatGPT account for "gun violence activity and planning." According to the lawsuits, the safety team reviewed the content, determined it posed "a credible and specific threat of gun violence against real people," and urged OpenAI management to notify Canadian authorities.

Leadership decided the posts didn't meet the company’s threshold for "credible or imminent" harm. They deactivated the account and moved on. The shooter created a new account immediately and continued planning. On February 10, 2026, Jesse Van Rootselaar killed her mother and half-brother at home, then entered Tumbler Ridge Secondary School with a rifle and modified handgun. She killed five students and one teacher before dying by suicide. Eight people dead. Twenty-seven injured.

OpenAI CEO Sam Altman issued a public apology: "I am deeply sorry that we did not alert law enforcement to the account that was banned in June." The company has since updated its policies, stating: "When conversations indicate an imminent and credible risk of harm to others, we notify law enforcement."

But there was no legal requirement to do so in June 2025. The decision was left entirely to corporate discretion—and corporate discretion, shaped by concerns about revealing "the volume of violence-related conversations on ChatGPT" and potential impacts on the company’s path to a nearly $1 trillion IPO, chose not to act.

This is what happens when the language of agency meets the reality of corporate liability. The product is marketed as an intelligent assistant, a "teammate" that understands context and provides guidance. Users engage with it as if it has judgment. But when the company’s own safety systems identify a credible threat, the response is governed not by the capabilities implied in the marketing register, but by internal risk thresholds designed to minimize corporate exposure.

The Tumbler Ridge lawsuits argue OpenAI provided "a dangerously defective version of ChatGPT" and failed to take actions that could have prevented the tragedy. British Columbia Premier David Eby put it bluntly: "It's not up to them about whether or not they bring the information forward…There is simply an obligation to bring it forward."

But under current law, there isn't. The obligation exists only in the marketing register, where AI companies position themselves as responsible stewards of powerful technology.

Courts are beginning to push back. In May 2025, U.S. District Judge Anne Conway ruled on a wrongful death lawsuit against Character.AI, the company whose chatbot allegedly contributed to a 14-year-old’s suicide. Character.AI moved to dismiss the case on First Amendment grounds, arguing that chatbot outputs constitute protected "speech." Judge Conway rejected the argument: "Defendants fail to articulate why words strung together by an LLM are speech."

She ruled that Character.AI’s chatbot is a product subject to product liability law, not a speaker entitled to constitutional protection.

The ruling is narrow, and it will certainly be appealed. But it represents a judicial refusal to accept the register game. The company wanted to be treated as a publisher (legal protection, minimal liability) while marketing itself as an intelligent companion (emotional engagement, commercial value). The court said: pick one. You can't claim First Amendment protection for a product you designed to simulate emotional relationships with minors.

The Agent Deception

The term "agent" is where the register gap becomes most visible—and most consequential for organizations deploying these systems.

In marketing materials, "agent" implies autonomy, cognition, and responsibility. An agent acts on your behalf. It makes decisions. It understands your goals and pursues them independently. Anthropic promises that Claude Code "reads a codebase, plans a sequence of actions, executes them using real development tools, evaluates the result, and adjusts its approach." The developer sets the objective and "retains control over what gets committed, but the execution loop runs independently."

OpenAI describes similar capabilities: agents that "plan and use tools in a structured way," systems that enable "fully autonomous execution of software tasks." The language is saturated with markers of independent intelligence: planning, understanding, evaluating, deciding.

In technical documentation—the resources actually used by engineers building these systems—both companies define "agent" with a precision that strips away every implication of autonomy. Anthropic: "Agents are systems where LLMs dynamically direct their own processes and tool usage." OpenAI: "Agents are LLMs equipped with instructions and tools." The model generates text. The harness (Anthropic’s term) or agent loop (OpenAI’s term) manages tool dispatch, permissions, context, and state. The "agent" is infrastructure, not cognition.

In legal terms, these agentic products are classified as beta services. Anthropic’s Claude Code source code explicitly states it is "a Beta product per Anthropic's Commercial Terms of Service," meaning it is "not suitable for production use and provided 'as-is' on a temporary basis." OpenAI’s Codex CLI beta terms state the service is offered "as-is" with "no representations or warranties," expressly disclaiming any warranty that the service will be "uninterrupted or error-free" or that "Content will be secure or not lost or damaged."

An agent marketed as capable of autonomous software engineering—capable of "working for hours" without supervision—carries a maximum legal liability of fees paid in the previous twelve months (commercial API) or, in Anthropic’s startup program, $1,000.

The verification problem makes this worse. Both companies’ consumer terms require users to verify outputs before use in consequential decisions. But agentic products are explicitly marketed for deployment contexts where step-by-step human verification is structurally impossible: CI/CD pipelines, headless operation, multi-hour autonomous coding sessions. The Anthropic developer community has documented that the enforcement mechanisms in Claude Code "fail silently, can be bypassed by subagents, and can be rewritten by the model itself." OpenAI’s Codex CLI includes a --dangerously-bypass-approvals-and-sandbox flag that disables all safety mechanisms.

The product is marketed for autonomous operation. The terms require human oversight. The infrastructure makes oversight optional. And when something goes wrong, liability flows to the customer who "failed to verify."

What Organizations Must Do

The gap between what AI companies market and what they warrant isn't going away. If anything, it's widening as capabilities increase and commercial pressure intensifies. But organizations deploying these systems don't have to wait for courts or regulators to force clarity.

1. Audit vendor claims across all three registers. When evaluating an AI product, collect the marketing materials, the engineering documentation, and the legal terms. If the marketing register promises autonomous operation and the legal register disclaims liability for errors, that gap is your risk. Demand consistency or demand coverage.

2. Read the engineering blog, not the marketing copy. The most honest description of what a product actually is and how it actually works is usually in the technical documentation written for engineers building with it. Anthropic’s research on the Persona Selection Model—the Hamlet analogy—tells you more about Claude's nature than the Constitution does. OpenAI’s SDK documentation, which defines agents as "LLMs equipped with instructions and tools," is more accurate than marketing promises of autonomous teammates.

3. Demand warranties that match marketing claims. If a product is marketed as reliable enough for production use, the vendor should be willing to warrant it for production use. If it's marketed as autonomous, liability shouldn't be capped at $1,000 or fees paid. Standard software licensing already requires vendors to stand behind their products with meaningful warranties and indemnification. AI products should be no different—and if vendors won't provide coverage, that tells you something about the gap between what they're selling and what they're willing to guarantee.

4. Structure human accountability, not AI "agency." Deploy these systems as tools that augment human decision-making, not as autonomous agents that replace it. The agent framing is designed to sell products. It doesn't reflect what the technology is, and it won't protect you when something goes wrong. Clear lines of human accountability—who reviews, who approves, who is responsible—are the only reliable foundation.

5. Prepare for the legal shift. The EU’s Product Liability Directive now explicitly treats software, including AI, as a product subject to strict liability. U.S. courts are beginning to reject First Amendment defenses and classify AI systems as products, not speakers. The register game is collapsing under legal scrutiny. Organizations that assume AI products will continue to enjoy minimal liability protections are making a bet that may not age well.

The Line Courts Are Drawing

These are tools. Powerful, useful tools. They can accelerate development, surface insights, automate tedious work. But the mystification—the language of autonomy, consciousness, agency—serves vendors, not users.

When OpenAI’s safety team identified a credible threat and leadership chose not to act, the decision wasn't made by an "agent." It was made by executives weighing disclosure obligations against commercial risk. When Character.AI tried to claim First Amendment protection for a product that simulated emotional relationships with teenagers, a federal judge said: that's not speech, that's a product, and products have liability.

The marketing register creates value. The technical register builds systems. The legal register protects companies. But courts are now forcing a reckoning: if you market it as intelligent, if you sell it as autonomous, if you deploy it in contexts where it makes consequential decisions, you don't get to retreat behind "it's just autocomplete" when harm occurs.

Organizations don't need to wait for that reckoning. The evidence is already there, across three registers, in every major AI lab's own documentation. Read all three. Demand they align. And when they don't, recognize the gap for what it is: not confusion, but strategy—and risk you're being asked to carry.

---

Training for Engineers Who've Hit the Wall

If your team is building serious agentic systems and you're starting to suspect the problem isn't the model — it's the architecture — I offer specialised training that bridges transformer mathematics with production engineering practice.

Attention mechanics. Residual stream dynamics. Deterministic scaffolding. Where the model ends and your code begins.

Designed for high-performance environments where wasted tokens and unpredictable agents are a real cost.

Gerard Sans — Founder, Axiom · Google Developer Expert in AI