AI incidents — what counts and what to log

What actually counts as an AI incident in 2026 — with worked examples, the EU AI Act Art. 73 serious-incident test, and the log structure reviewers expect.

What counts — worked examples

  • Hallucination with consequence — a customer-support LLM tells a buyer your warranty covers something it does not, and you honour it. Yes, log it.
  • Prompt injection — a CV uploaded to your ATS contains hidden instructions that bias the screening LLM. Yes, OWASP LLM01:2025.
  • Sensitive data in prompts — an employee pastes a customer list into a free ChatGPT account. Yes — GDPR personal-data incident regardless of harm.
  • Bias drift — quarterly fairness test shows the model now under-approves a protected group by 11pp vs baseline. Yes, log and remediate.
  • Agent runaway — an autonomous agent makes 400 API calls in a loop and bills $9k. Yes, governance and cost incident.
  • Vendor breach — your LLM vendor discloses a training-data exposure that includes your prompts. Yes, and notify customers per your DPA.

The minimum log structure

  • Incident ID, date detected, date occurred (often different).
  • System ID, model version, prompt template version.
  • Severity (Low / Med / High / Serious per EU AI Act Art. 3(49)).
  • Affected parties and data categories.
  • Root-cause classification (model, data, prompt, user, vendor, agent).
  • Containment actions, remediation, owner, due date.
  • Regulatory clock: GDPR 72h, EU AI Act 15 days for serious, narrower for fundamental-rights breach.

Do this Monday

  1. Create an "AI incidents" Jira / Linear project with the schema above as the issue template.
  2. Wire your AI gateway (LiteLLM, Portkey, Azure AI Content Safety) to auto-open a ticket on safety filter triggers.
  3. Run a tabletop on the prompt-injection example with engineering + legal.

Reviewer hot-buttons

  • A log that shows zero incidents is itself a finding — it means you are not detecting them.
  • Severity must be defined; "serious" must reference EU AI Act Art. 3(49) wording.
  • Closure must show what changed, not just "fixed".