Australia's Robodebt scheme wrongly pursued approximately 400,000 welfare recipients and unlawfully recovered around 1.76 billion AUD because automated debt notices issued at scale with no officer authorising any individual determination before it was sent.
Between 2015 and 2019, the Australian government ran a welfare compliance programme that came to be known as Robodebt. It used automated income averaging to calculate alleged overpayments and issue debt notices to people who had received benefits. The design step that defined the scheme was the removal of human review. Notices went out unreviewed, generated and sent by the automated process rather than assessed by an officer before issue.
The averaging method was crude. It spread reported annual income evenly across fortnightly periods and treated the result as proof of a debt, shifting the onus onto recipients to disprove figures the system had produced. The scheme wrongly accused about 400,000 people and unlawfully recovered approximately 1.76 billion AUD before it was wound back.
A Royal Commission reported in 2023 and found the scheme crude, cruel, and unlawful, as summarised by the Law Society Journal and analysed by the Blavatnik School of Government at Oxford. It stands as one of the largest documented failures of automated government decisioning.
What actually failed - the governance gap
The central failure was structural, not technical. A consequential action, asserting that a citizen owed a debt to the state, was allowed to run end to end with no named human standing between the calculation and the notice. The system that computed the figure was also the system that issued the demand. Nothing required an officer to authorise each determination before it left the building.
That single design choice removed three things at once. It removed accountability, because no one signed any individual notice. It removed segregation of duties, because the calculator and the issuer were the same automated path. And it removed the friction that would have surfaced the flawed averaging method early, because a person reviewing determinations one by one would have seen the same implausible debts the recipients later disputed.
The legality of income averaging was a separate question, eventually answered by the courts and the Commission. The governance question is narrower and was the one that turned a flawed method into a mass event. An irreversible, harmful action was permitted to issue at scale with no authorisation step and no per-decision record of who stood behind it.
How MakerChecker changes the outcome
MakerChecker governs what an automated actor is allowed to do, not the quality of the calculation behind it. The Robodebt pattern maps onto a small set of its capabilities.
Deny by default means a debt notice cannot issue unless the action is explicitly granted. The role that runs the averaging calculation can be granted the right to propose a determination and nothing more. Issuing the notice is a separate, higher-risk skill that the calculating role does not hold.
role: debt-calculator
grants:
- skill: debt.assess
version: "1.0"
risk_tier: low # may propose, may not issue
The issuing action routes through an approval gate. Each debt determination parks and waits for a named officer to sign before any notice is sent. Routine proposals accumulate; the irreversible step, the demand against a citizen, runs only after authorisation.
skill: debt.issue
risk_tier: high
gate:
approvals_required: 1 # named officer must sign each determination
forbid_requester: true # calculator cannot authorise its own output
Segregation of duties through forbid_requester keeps the system that produced
the figure from approving its own demand. The calculator proposes; a separate
authorised human disposes. Every grant check, every proposal, every approval,
and every denial is written to the tamper-evident, Ed25519-signed, hash-chained
audit. The result is a per-debt record of who authorised each notice, verifiable
offline, rather than a population of demands no one signed.
The effect is not only accountability after the fact. A gate that forces an officer to authorise each determination puts a person back in front of the implausible numbers. That is the friction Robodebt removed, and it is the friction most likely to surface a flawed method before it reaches 400,000 people.
What MakerChecker would not fix
MakerChecker would not fix the income averaging itself. It does not check whether a calculation is sound, and it would not have told anyone that spreading annual income across fortnights was the wrong method. Nor does it answer the legal question the courts and the Royal Commission settled. A gate can route a flawed determination to a human, but it cannot make the determination correct.
What it changes is the structure around the action. It forces a named officer to authorise each notice, refuses issuance without that authorisation, separates the calculator from the issuer, and leaves a signed record per debt. Those controls do not repair the method. They make the harm visible and accountable, and they restore the human review step whose removal was the defining feature of the scheme, which makes the underlying problem far more likely to surface early.
See the configuration: examples/rogue-ai/australia-robodebt-automated-debt-recovery