Who Answers for the Mistake
The model makes the mistake, and the responsibility sits with the person or the company using its answers. That is how it went in three proceedings that have already concluded.
The chatbot promised a discount
A passenger asked the chatbot on an airline's website about the fare for people flying to a funeral. The bot said the discount could be claimed retroactively within 90 days. Company policy did not allow that.
The passenger bought a full-price ticket, filed a claim, and was turned down. He then took the case to the Civil Resolution Tribunal of British Columbia, which handles small civil disputes online and is not part of the court system.
In February 2024 the tribunal ordered the airline to pay the difference. The company's argument that the bot answers for itself was rejected: the bot sits on the company's website, and the company is responsible for what its website says.
Six decisions that never existed
In 2023 a lawyer filed a brief in a US court. To support his position he cited six decisions that courts had supposedly issued in similar disputes. ChatGPT had found those decisions for him, and not one of them existed.
The judge imposed a $5,000 sanction on the lawyers and their firm. Separately, he ordered them to send letters to the real judges whose names appeared in the invented decisions.
A report with sources that do not exist
In 2025 a consulting firm delivered a report to an Australian government department for 440,000 Australian dollars. It was found to contain references to publications that do not exist and an invented quotation from a court decision.
The firm acknowledged that part of the text had been prepared with a language model and refunded more than 97,000 Australian dollars.
What this means for you
The signature on the document is yours. A model does not sign emails, reports, or memos, and no proceeding will ask it anything.
Check what leaves your hands. Numbers, dates, quotations, and references to documents and laws get verified in the original source. How to do that is described in the chapter "Mistakes and Invented Facts".
Say what you used, if that is the practice at your company. In all three cases the trouble started not with the fact that a model had been used, but with the fact that nobody checked the mistake.