Independent directory.  Not affiliated with or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic.
← All articles

Risk & governance

Who is liable when AI gets it wrong?

In practice, you are. A customer harmed by a wrong answer, a regulator asking why a decision was made, an auditor querying a posted journal — all of them come to the business that deployed the system, not to the model provider and usually not to the consultant who built it. What the contract and the design can do is decide who bears the cost afterwards, and whether you can explain what happened at all.

The short version

Deployment is what attracts liability, so the deploying business carries it by default. Three things change your position materially: a human approval point on consequential actions, an audit trail that can reconstruct any decision, and a contract that says what the builder warranted. None of these is a technical feature you buy — they are design decisions taken before the build.

This is general information about how these arrangements are usually structured, not legal advice. Anything consequential needs a lawyer in your own jurisdiction.

Why it lands on the deployer

Because that is how responsibility for a tool generally works. If you use a spreadsheet with a broken formula to price a contract, the customer’s claim is against you, not Microsoft. An AI system is not obviously different in principle: you chose to deploy it, you decided what it was allowed to do, and you presented its output as your own.

Model providers disclaim extensively in their terms, and consultants limit liability in theirs. Both are normal and neither is unreasonable — but the combined effect is that the risk settles on the business at the end of the chain, which is the one whose name is on the output.

Does a human approval point change anything?

Substantially, and it is the single most effective control available. A system that recommends, with a person approving, is a decision-support tool and the approver is accountable in the ordinary way. A system that acts without review is making the decision, and the organisation is answering for outputs nobody examined.

The distinction only holds if the approval is real. A screen where a human clicks accept forty times an hour without reading is not an approval step, and it will not read as one to anyone investigating later. Meaningful review means sampling rates that a person can actually sustain, the evidence presented alongside the recommendation, and the ability to disagree without it being difficult.

This is why the approval design belongs in the specification and not in a later phase. It determines what kind of system you have built, in a sense that matters outside engineering.

What does an audit trail actually need to contain?

Enough to reconstruct a single decision months later, when the person who built the system has moved on and the model version has changed. In practice:

Without that, “we don’t know why it did that” is your entire answer, and it is not one that survives contact with a regulator, an insurer or a claimant. The platform-level controls that now exist — session transcripts, activity feeds, per-tool permissions — make this easier to assemble than it was, but assembling it is still a decision someone has to take.

What should the contract say?

Five clauses do most of the work, and every one of them is easier to agree before the engagement than after an incident:

What about professional indemnity insurance?

Ask whether the practitioner carries it and whether their policy contemplates this kind of work. Many professional indemnity policies were written before this category existed and may exclude it, or simply not address it. A practitioner who has checked with their insurer and can say so is demonstrating something useful about how they operate, independent of the answer.

The practical position

You cannot contract your way out of being the deployer. What you can do is make the failure explicable and the exposure bounded: put a real human decision on anything consequential, log enough to reconstruct any single output, agree the threshold and the warranty in writing, and size the engagement so the cap is not absurd relative to the risk.

Every one of those is decided in the first fortnight, by whoever is scoping the work. It is worth asking a prospective practitioner how they handle all four before asking what they charge — the answers separate people who have deployed something inside a regulated business from people who have not.

Does it matter whether we built it or bought it?

Less than people expect. Buying a product shifts some responsibility to the vendor through their warranties, but you still chose to deploy it, configured what it could reach, and decided what it was allowed to do unsupervised. Those decisions are yours in either case.

Where it does matter is evidence. A bought product usually comes with documentation about intended use and limitations, and deploying it outside those limits is a specific, demonstrable failure on your part. Read them, and record that you did.

What if the consultant made the mistake?

Your customer still comes to you, and you then look to your contract with the consultant — which is why the warranty and liability terms matter far more than they appear at signature. Two separate questions arise: whether they breached what they promised, and whether the cap on their liability leaves anything worth recovering.

Fee-value caps are common and reasonable from the supplier’s side, and they mean a modest project cannot indemnify a large loss. That is not a reason to demand unlimited liability, which competent practitioners will decline. It is a reason to size engagements so the potential loss and the available recovery are not wildly out of proportion, and to keep humans on the decisions where they would be.

How does this apply to autonomous agents?

It sharpens everything above, because the human approval point — the strongest control available — is the thing being removed by design. An agent that acts on its own is your organisation acting, at machine speed, on decisions nobody reviewed.

The compensating controls are scope and evidence. Constrain what the agent can reach and what it can do irreversibly; cap what it can spend; log every action in a form that reconstructs the sequence afterwards. Platform features now exist for each of those, but choosing the limits is a governance decision, not a configuration one, and it belongs to whoever would answer for the outcome.