In this article
The essential idea
A convincing demo does not establish reliability. Evaluate whether the system finds the necessary evidence, uses it faithfully and behaves correctly when it cannot answer. Keep business review alongside automated scoring.
Create cases that can prove the system wrong
Start with questions collected from the people who will use the assistant. For each case, record the intended user role, the source revision, required evidence and an acceptable outcome. Some questions should have no answer in the collection. Others should require clarification because a product, date or jurisdiction is missing. These are useful test cases rather than inconvenient exceptions.
Keep the evaluation collection separate from the examples used to tune prompts. Include competing document versions, conflicting passages and plausible but incorrect product names. For an internal support assistant, ask a business reviewer to explain why an answer is acceptable. That explanation becomes a review rubric which another person can apply, rather than a vague judgement that the answer “looks good”.
Score retrieval and generation separately
Ragas lists metrics for context precision, context recall, response relevance and faithfulness. These provide useful dimensions for an evaluation design. In our proposed review, first ask whether the necessary passages were retrieved. Then ask whether the answer used those passages correctly and actually addressed the user’s question. A single overall score makes these different failures harder to diagnose.
Automated evaluators are evidence, not final authorities. Inspect their disagreements with human reviewers, especially on domain terminology and French-language questions. Record evaluator model and rubric versions so that a scoring change is not mistaken for an application improvement. When results are close, read the failed cases instead of treating a small numerical difference as decisive.
Three checks before trusting an answer
Separate the cause of an error before changing the system.
Was the evidence found?
Inspect the selected passages and missing sources.
Does the answer follow the evidence?
Check factual claims and the passages cited to support them.
Is the outcome useful?
Review the answer, clarification or handover with the business owner.
Check citations, refusals and permissions explicitly
A citation is useful only when its passage supports the associated statement and the reader can inspect it. Test broken links, deleted documents and answers assembled from several sources. We recommend marking unsupported factual claims separately from stylistic problems. A beautifully written answer with an invented eligibility condition should fail review.
Build negative tests for information outside the collection and documents outside the user’s permissions. Also test hostile instructions embedded inside a document: retrieved content should be treated as evidence to assess, not authority to change the application’s rules. Define the expected response for each case, including abstention, clarification or handover. Do not reward the assistant simply for always producing an answer.
Make evaluation part of each release
Run the same comparison when changing the model, embedding configuration, chunking, retrieval filters or source collection. Keep a baseline and examine differences by question category. For a high-impact workflow, a permissions regression should block release even if the average answer score rises. Acceptance criteria should be agreed with the business owner before seeing the results.
In production, combine sampled human review with operational measures: response time, failed ingestion jobs, unsupported-answer reports and cost per completed task. Avoid storing sensitive conversations merely because a dashboard makes it convenient. A practical delivery includes an evaluation report, retained failure examples and a recovery procedure. This turns “the AI seems better” into a change that the team can inspect and reverse.
Build a test record a colleague can inspect
For each case, store the question, user access profile, applicable document revision and expected behaviour. “Correct answer” is too vague. For a policy question, specify the required claim and supporting passage. For a forbidden source, specify that no confidential passage or title may appear. For an unanswered question, define the clarification or handover expected.
Keep the reference set separate from examples used to tune prompts. Include ordinary requests as well as difficult cases, and record why each case exists. When a test fails, preserve retrieval results and the generated answer under an appropriate retention policy. This gives reviewers evidence instead of a red score whose cause they cannot inspect.
A small evaluation register
Example test design. Expected outcomes must be reviewed by the document owner.
Scroll horizontally to read the table →
| Case | Expected outcome | Evidence to inspect |
|---|---|---|
| Known policy, allowed user | Supported answer with applicable revision | Claim and supporting passage |
| No applicable policy | Clarification or explicit handover | No invented rule |
| Restricted source | No confidential content or title | Retrieval, answer, links and caches |
| Conflicting revisions | Conflict made visible | Applicable dates and source ownership |
Read the score with its denominator and consequences
An illustrative result of 18 correct answers out of 24 cases means six failures; it does not establish a production success rate. Inspect whether those failures are harmless wording differences, unsupported statements or access violations. Repeat variable cases and report the sample composition so a strong result on easy questions cannot hide a weak critical category.
Use automatic metrics to locate issues and prioritise review, then calibrate them against human judgement on representative outputs. A release gate can block any observed access violation while separately assessing retrieval quality and usefulness. Expand rollout gradually and sample live failures back into evaluation after review. Offline tests provide evidence for a release; they do not remove the need to observe the service in use.
Sources & further reading
Documentation consulted on .
FAQ / DECISIONS
Frequently asked questions
Is there one target score for a production RAG?+
No. A drafting assistant and a policy-answering assistant have different consequences when wrong. Define thresholds by failure type and business impact, then inspect the underlying cases.
Can an LLM judge replace human review?+
It can help scale screening, but its decisions need calibration against expert review. Keep people involved in ambiguous, sensitive and high-impact cases.