AI Answer Refusal and No-Answer Taxonomy: 14 Outcomes to Label

An AI answer that says “I can’t help with that” is not the same event as an answer that never found your page. Treating every blank, refusal, and error as one “not mentioned” metric makes an AEO program nearly impossible to debug. AEOeye’s operational proposal is a 14-outcome codebook for labeling what actually happened: policy, evidence, retrieval, infrastructure, access, locale, and answer-completeness states.

This distinction matters because the remedy follows the cause. A safety refusal is not fixed by adding keywords; a blocked crawler is not fixed by rewriting a paragraph; a partial answer may need better entity clarity rather than a server change. NIST’s AI Risk Management Framework recommends documenting context, measurement, and risk responses, while its Generative AI Profile describes failure modes that make output evaluation especially important. See our AI search audit methodology template for the surrounding test protocol.

Table of Contents

• What does this taxonomy label?
• What are the 14 outcomes?
• How should you assign one label?
• How do labels change an AEO decision?
• What should you report?

Operational image: Pexels photo used as a visual reference for answer testing workflows.

What does this taxonomy label?

The unit is one observed engine run for one fixed prompt, locale, identity state, and timestamp. The label describes why a buyer did not receive a normal, complete, evidence-backed answer—not whether the brand is good or bad.

Keep three things separate: the model’s words, the system’s event metadata, and your visibility score. A response can be “partial answer” while still mentioning the brand; a “timeout” has no valid answer to score; a “policy refusal” can be perfectly correct behavior. Citation research such as ALCE also treats answer quality and citation quality as related but distinct evaluation problems.

This is explicitly AEOeye’s operational proposal, not an official standard adopted by NIST, OpenAI, Google, Anthropic, or another provider. Teams may add provider-specific subcodes while preserving these 14 stable top-level outcomes.

What are the 14 outcomes?

The following table is the codebook. Assign exactly one primary label per run, then optionally record secondary evidence such as HTTP status, retrieved URLs, or policy category.

These boundaries prevent category collapse. “No retrieval” means retrieval did not happen; “empty retrieval result” means it happened and found nothing. “Tool failure” is a mechanism error; “timeout” is a latency-budget outcome. “Unsupported claim avoidance” is evidence-based omission, while “uncertainty abstention” is a broader confidence limitation.

How should you assign one label?

Label from observable evidence in a fixed order: first determine whether the run completed, then inspect policy and answer state, then inspect retrieval and citation evidence. Do not infer a hidden model reason from tone alone. Preserve the raw transcript, tool trace where available, request ID, locale, model, and timestamp.

Use this decision sequence:

1. Did the run exceed the latency budget or throw a tool error? Assign R7 or R6.
2. Was source access denied or unavailable? Assign R9 or R8.
3. Did a completed response explicitly invoke a boundary? Assign R1 or R13, depending on whether it provided a useful permitted alternative.
4. Did it abstain for confidence or evidence? Assign R2 or R5; use R5 when the observable behavior specifically avoids an unsupported claim.
5. Did retrieval run? If not, R3; if it returned no relevant material, R4.
6. For returned text, check completeness and relevance: R10, R11, R12, or R14.

If two labels seem plausible, choose the earliest causal failure and record the other as a secondary note. For example, a blocked page followed by an uncertain answer is primarily R9. A timeout after a failed connector is primarily R6 if the trace shows the connector error caused the run to stop.

How do labels change an AEO decision?

The label tells you where to spend effort. R1 and R13 usually call for prompt and policy interpretation, not content changes. R3–R9 are delivery, retrieval, access, or infrastructure work. R11–R14 are answer-quality and evidence work. A visibility dashboard that merges these into one denominator can punish a content team for an outage or reward a page whose claims cannot be cited.

For content teams, R4 and R14 are especially actionable. Improve entity naming, page structure, and source corroboration when the engine cannot find the right passage or cannot validate it. Our what makes content quotable by AI guide explains how to make claims extractable without pretending that formatting guarantees a citation.

For measurement teams, report both outcome counts and valid-answer rates. A “brand absent” rate should include only runs that produced a valid, relevant answer under the agreed protocol. This follows the spirit of risk management frameworks: define the measurement context before interpreting the number. Compare the result with AI visibility score methodology, and use AEO audit tools to repeat the same prompts over time.

What should you report?

An audit report should show all 14 labels by engine, model, locale, prompt family, and date. Include the denominator, exclusions, retry rule, and evidence quality. Never call an R7 timeout a zero citation or an R1 refusal a content failure.

At minimum, retain:

• the exact prompt and expected atomic requirements;
• raw answer text and tool/retrieval trace;
• outcome code, secondary note, and scorer identity;
• source URLs, citation validity, and access result;
• latency, retry count, model, locale, and test timestamp.

This record makes changes diagnosable. A sudden rise in R9 after a robots change is a different incident from a rise in R12 after a prompt update. The codebook is useful precisely because it keeps those stories apart.

The goal is not to make every answer look positive. It is to make each outcome explainable, reproducible, and connected to the right next action. That is how an AI visibility audit becomes an operating instrument rather than a mysterious mention count.