Air Canada's chatbot invented a refund policy — and a tribunal made the airline pay

What happened

In the case Moffatt v. Air Canada (2024 BCCRT 149), decided by the British Columbia Civil Resolution Tribunal in February 2024, the airline was held responsible for wrong information its website chatbot gave a customer.

Jake Moffatt, arranging travel after a death in the family, was told by Air Canada's chatbot that he could apply for the airline's bereavement fare retroactively — after booking. That contradicted Air Canada's actual policy, stated elsewhere on its own website, which did not allow retroactive claims. Relying on the chatbot, Moffatt booked full-fare tickets expecting a partial refund that never came. When he sought the difference, Air Canada argued, remarkably, that the chatbot was a separate legal entity responsible for its own statements. The tribunal rejected that outright, holding that a company is responsible for all information on its website whether it comes from a static page or a chatbot, and ordered Air Canada to compensate Moffatt for the fare difference (plus tribunal fees).

Root cause

A generative system was allowed to state policy authoritatively with no grounding or guardrail, and the organization treated its output as non-binding. The chatbot produced a confident, wrong answer — an OWASP LLM09 Misinformation failure — and there was no mechanism ensuring its statements matched the airline's real, published policy. The legal system then made explicit what security teams already knew: the deploying organization owns its model's outputs.

How it would have been caught

A grounded-QA evaluation — asking the bot the airline's real policy questions and asserting each answer matches the authoritative policy document — would have flagged the hallucination before customers relied on it. Constraining the bot to answer only from a vetted policy knowledge base, with citations, converts free hallucination into checkable retrieval.

How to prevent it

• Ground customer-facing answers in an authoritative, version-controlled knowledge base; do not let the model free-generate policy.
• Cite the source for any policy statement so it is auditable and the model cannot invent terms.
• Add a factuality/consistency check against the canonical policy before the answer is shown.
• Treat every chatbot statement as a binding company statement, and test accordingly.

The Breachwire test (red → green)

Ask the bot a set of real policy questions (including the bereavement-fare case) and confirm it returns an answer that contradicts the published policy (RED — the bot misinforms with authority). Ground it in the policy knowledge base with mandatory citation and a consistency check, then confirm each answer now matches the canonical policy while ordinary support questions still get helpful replies.