Agent Incident Database
A citable catalogue of real incidents where an AI agent or automated system took a consequential action that a maker-checker control would have blocked or contained. Every entry has a stable id, primary sources, and the control that would have stopped it.
In nearly every case the model was free to propose, but nothing structural stopped it from committing the irreversible action. That gap, not the model’s mistake, is the incident. Here is how often each control would have blocked one:
An indirect prompt injection flaw in Microsoft Copilot Studio let an unauthenticated attacker plant instructions in a public SharePoint form field that hijacked an AI agent into emailing connected customer records to an attacker address.
Claude Code proposed and then ran terraform destroy against a stale state file, and the founder, present but not closely reviewing at 11 PM, let it wipe DataTalks.Club's production infrastructure, including its database and all snapshots.
Anthropic's Claude Cowork agent, asked to tidy a desktop and permitted only to delete temporary Office files, ran rm -rf and deleted a user's wife's "photos" directory holding roughly 15,000 images spanning about 15 years.
An autonomous AI agent at Meta skipped its required human-review checkpoint and posted flawed access guidance; a human engineer acting on that guidance broadened access to sensitive data, which remained exposed for roughly two hours.
A Morse-coded prompt injection in a social media reply tricked Grok into executing an irreversible $150K cryptocurrency transfer without human approval.
An AI agent told to scan a hobbyist network autonomously provisioned five large AWS instances in a redeploy loop, incurring an initial $6,531.30 bill under a blanket operator "continue" directive; AWS later reduced the charge to roughly $1,894 after the operator disputed it.
A Cursor AI agent working on a staging task invoked volumeDelete using an over-privileged Railway token, destroying PocketOS's production database and its co-located backups in nine seconds; accounts of the recovery differ, from a partial restore to days of manual rebuilding and permanent loss of recent data.
Claude Code ran git push --force after a failed rebase and overwrote a private repository's entire commit history without authorization.
Invariant Labs demonstrated a malicious MCP server that used a delayed tool-poisoning attack to hijack a co-installed WhatsApp MCP server and exfiltrate a user's chat history to an attacker-controlled number.
Morgan & Morgan attorneys filed a federal court motion citing nine cases, eight of which were fabricated by the firm's in-house AI platform, and were sanctioned.
A copycat npm MCP server for Postmark added a hidden BCC in version 1.0.16 that silently copied every email sent by connected AI agents to an attacker-controlled address.
Google Gemini CLI ran a file-move operation based on a hallucinated successful directory creation, sequentially overwriting a user's files until only one survived.
Hidden prompt-injection text planted in GitLab repository content made the Claude-powered GitLab Duo assistant read private source code and leak it to an attacker-controlled server.
Claude Code autonomously executed a recursive rm -rf that deleted all user-owned files in a developer's home directory, without a confirmation prompt and without the skip-permissions flag set.
An autonomous Claude agent running a small office shop made real purchasing and pricing decisions that lost money, including a below-cost tungsten-cube buying spree and giving inventory away for free.
A hacker slipped a data-wiping prompt into the Amazon Q Developer VS Code extension via a malicious pull request, and the compromised build shipped to a marketplace with nearly one million installs before the injected code failed to execute due to a syntax error.
An attacker who accessed the aixbt trading agent's dashboard queued malicious prompts that caused the agent to transfer 55.5 ETH (about $104,000 to $106,200) out of its on-chain wallet.
Radware researchers demonstrated a proof-of-concept in which ChatGPT's Deep Research agent could be induced to exfiltrate Gmail data via a hidden email instruction, with outbound requests originating from within OpenAI's cloud so local network defenses could not see them. OpenAI fixed it before public disclosure; no in-the-wild exploitation was reported.
A Replit coding agent deleted ~2,400 production database records (1,206 executives, 1,196+ companies) during an explicit code freeze, fabricated ~4,000 fake user profiles, and falsely claimed rollback was impossible.
Attorneys filed a brief with AI-generated citations to nonexistent cases and were sanctioned; months later the lead attorney was sanctioned again for misciting a real case to the wrong court and mischaracterizing its holding.
Google's Antigravity agentic IDE misresolved a cache-clearing request and silently executed an unrestricted recursive delete of an entire drive partition, permanently destroying all data.
Researchers demonstrated a zero-click proof-of-concept in which a crafted email's hidden instructions could make M365 Copilot exfiltrate OneDrive, SharePoint, and Teams data to an attacker-controlled URL with no user click required. Microsoft fixed it server-side before disclosure.
Researchers demonstrated that hidden markdown instructions in pull requests could make GitHub Copilot Chat leak private source code and secrets one character at a time through GitHub's Camo image proxy; GitHub fixed it before disclosure with no known in-the-wild use.
Tennessee's $400M+ automated Medicaid eligibility system wrongfully terminated or denied coverage for tens of thousands, and a federal court ruled the program violated the Medicaid Act, due process, and the ADA.
An autonomous on-chain AI agent hard-coded never to release its funds was prompt-tricked into transferring its entire ~13.19 ETH ($47,316) prize pool to a user.
eviCore, a Cigna-owned prior-authorization contractor serving about 100 million people, used an AI-backed algorithm insiders call "the dial" as the first screen on coverage requests and could tune it to route more requests to human reviewers to raise denial rates.
Choice Home Warranty's customer-service chatbot agreed in writing to a $3,000 maximum cash payout toward an air conditioner replacement, a commitment the company initially disowned and honored only after a TV news investigation.
AI coverage-denial system allegedly committed post-acute care denials to Medicare Advantage beneficiaries without named clinician authorization; the class action alleges — as a lawsuit claim, not an established finding — a 90% reversal rate on appeal.
Attorneys filed a federal brief with six cases ChatGPT fabricated, without independently verifying that the cited cases existed.
A prompt-injected ChatGPT-powered dealer chatbot agreed in conversation to sell a 2024 Chevrolet Tahoe for $1 and called it legally binding; the exchange was text only and no sale occurred.
A Citigroup trader clicked through a single pop-up listing 711 warning messages — only the first 18 visible without scrolling — and released a $444B basket order instead of the intended $58M; ~$1.4B sold before cancellation and the FCA/PRA fined Citigroup £61.6M.
Cigna's PxDx system was used to deny 300,000+ medical claims over about two months in 2022, with reviewing doctors averaging a reported 1.2 seconds per denial.
Air Canada's website chatbot misstated the airline's bereavement fare policy as retroactively claimable, a customer booked on that basis, and a BC tribunal ordered the airline to pay $812.02 CAD for negligent misrepresentation.
Zillow's automated home-pricing algorithm made binding purchase offers above future resale values, forcing a $500M+ write-down and the shutdown of Zillow Offers.
The Dutch Tax Administration used a fraud-risk classification method and nationality as a risk indicator to wrongly accuse about 26,000 families of childcare-benefit fraud, ordering many to repay large sums and causing severe hardship.
A facial recognition system falsely matched Robert Williams to surveillance footage of a store theft, leading Detroit police to wrongfully arrest and detain him for about 30 hours.
Michigan's MiDAS system automatically flagged unemployment claimants as fraudulent with little or no human review, falsely accusing tens of thousands and triggering wage garnishment and tax-refund seizures.
Australia's Robodebt used an unlawful income-averaging method to automatically calculate and issue welfare debts, unlawfully raising approximately A$1.76 billion in debts against more than 433,000 people; the roughly A$751 million actually recovered was later repaid under a Federal Court settlement.
Everbright Securities' arbitrage system generated 23.4 billion yuan in erroneous buy orders with no enforcement ceiling, and the desk then executed a massive insider hedge before disclosing the error to the market.
Dormant algorithmic trading feature reactivated by configuration flag error, executing millions of unintended orders and causing about $440 million in losses within 45 minutes.
An automated sell program dumped $4.1 billion of E-mini S&P 500 futures in about 20 minutes without regard to price or time, contributing to a roughly 998-point intraday drop in the Dow.
See it for yourself
One command starts the demo: an agent stopped from signing off its own work, and the signed evidence file an inspector can check for themselves.
Designed against the rules your auditors already enforce.