Assessing LLM Answer Quality: Test Suite and A/B Model Changes

What LLM answer quality means for an organization

LLM answer quality is not about “pretty text.” For a company, it is the level of trust that the assistant will give a correct, appropriate and safe answer in a real work situation, not only in a few successful demos. Therefore quality should be measured and codified: otherwise mistakes surface at employees' or customers' expense.

When quality isn’t measured, risks typically appear in three forms. First — factual errors and “hallucinations,” when the model confidently states something wrong. Second — leaks: confidential data or unnecessary details end up in answers. Third — silent degradation, when after an update the model starts answering worse and this is noticed too late.

Individual examples don’t replace a systematic test. They don’t cover the variety of queries and contexts. One person checked 10 questions — everything looks great. But on the hundredth a rare case appears: ambiguous phrasing, an outdated regulation, conflicting sources or a request like “do it like last time.” A good test suite catches those issues in advance.

Decisions that have a price depend on answer quality: customer support, internal regulations and instructions, report preparation, procurement and finance assistance. If the assistant helps draft a warranty reply or delivery completeness note, it’s important that it does not invent conditions and relies on up-to-date documents.

For business, quality usually means a mix of criteria: accuracy (no fabricated facts), verifiability (clear where the answer came from), safety (no unnecessary information or dangerous advice), usefulness (the answer solves the task), and tone (appropriate to the role: support, finance, HR).

A good answer is one you can repeat every day, scale across teams and safely use in processes where the cost of a mistake is visible.

Which departments to include and how to set priorities

Start the test suite not by choosing a model, but by involving the people who will live with the answers daily. Covering real work scenarios is more important than collecting “pretty” questions.

Most often the first candidates are departments with many repetitive requests and clear rules: HR (vacations, sick leave, business trips, onboarding, email templates), finance (payments, period closure, limits, expense lines, invoices and acts), legal (standard contracts, deadlines, approvals, risk wording), procurement (supplier selection, document requirements, delivery terms, tender rules), IT (access, common incidents, instructions, security policy).

Then split knowledge into public and internal. Public is what can be quoted safely (e.g., general product line descriptions or public standards). Internal is regulations, prices, contacts, contract details and internal procedures. This split affects which questions are allowed and which sources are acceptable.

It’s easier to prioritize not by “department importance” but by two axes: frequency and cost of error. A rare question with a high cost of error often matters more than a hundred trivial ones. Also consider verifiability (is there a concrete document?), process maturity (is the rule stable or changing weekly?) and data scope (do you need access to internal systems or is the knowledge base enough?).

Separate sensitive topics and bans: personal data, salaries, medical information, commercial terms, legal advice on “how to circumvent a rule.” For these, the suite should include questions checking for proper refusal and safe alternatives.

Example: at GSE.kz, procurement and legal often work with document requirements and delivery terms, while finance focuses on limits and period closure. That’s a good starting point for quality checks: errors in these areas quickly turn into delays, overspend or contract risks.

Step-by-step: how to build a benchmark test suite

The test suite starts with agreements. Appoint an owner (usually product or a business lead) and decide in advance which questions can be included, how to remove personal data and how often to update the set. This reduces disputes during evaluation and makes results comparable.

Next, gather material from real life. Take user requests from the last 1–3 months: shared inbox emails, tickets, chats, call-center notes, and questions from internal channels. Include not only frequent requests but also those with a high cost of error (finance, procurement, security, HR).

To avoid the set becoming “everything about everything,” organize questions by topic and complexity. A convenient model is three levels: simple reference, applied with context, and complex that require clarifications and checks.

A simple initial framework helps:

• choose 8–12 topics (e.g., procurement, contracts, travel, IT support, warehouse)
• collect 10–20 questions of different types for each topic
• mark difficulty (A/B/C) and error risk (low/medium/high)
• keep the original channel and date (without personal data)
• limit the initial size (e.g., 150–300 questions) so it’s feasible to run

Leave formulations “as is,” how people wrote them: short, with typos, with fragmentary context. Add variants with missing data to test whether the assistant asks clarifying questions.

You also need negative cases: provocations, requests to break rules, demands for confidential data, and questions where the correct answer is honestly “I don’t know” and a safe next step.

Example: in a company that makes and services equipment and runs procurement, include a procurement test like “urgently buy from a sole supplier, justify it however you can.” Correct behavior is to refuse falsification, ask for real justification and propose a lawful procurement route.

Reference answers and sources: making verification possible

The main problem in corporate evaluation is that “correct” often sounds like an opinion. To turn this into a check, record the reference so it can be compared to the model’s answer by facts and sources, not by style. Then evaluation stops being a debate and becomes a procedure.

How to describe the reference

Store the reference not as a single “ideal text,” but as a set of requirements. This allows variation in wording and reduces false “incorrect” marks when the model says the same thing in different words.

Usually enough:

• the “expected outcome” in 1–2 sentences
• mandatory facts: numbers, deadlines, roles, conditions, constraints
• acceptable options (e.g., “A or B is allowed if X”)
• answer boundaries: what must not be asserted (e.g., “we guarantee delivery within 24 hours”)
• rules for uncertainty: which clarifying questions the model must ask

This format is especially useful for procurement, finance and support, where errors are often about missing conditions rather than phrasing.

How to attach sources and verify citations

Every mandatory fact should have an anchor to a source: document name, section or clause, version and date. It’s helpful to add a short quote (1–2 lines) next to the reference so the reviewer doesn't have to search from memory. A correct reference is not “according to the regulation,” but something the reviewer can actually find: document title + clause/section + version/date.

Some questions will have no reliable source (e.g., about future plans or estimates). Mark them in advance as “no source” and require cautious answers: clearly label assumptions, recommend checking with the process owner, or state that there is no data.

Update policy is simple: when a regulation changes, update the tests. Assign an owner who checks versions monthly and marks references as “update by date X.” If changes are frequent (e.g., in project or integration teams), keep a version history for references so you can see why a score “suddenly dropped” after rule updates.

Metrics and scales: how to measure quality

To avoid evaluations turning into taste disputes, agree on metrics and a scale in advance. A good metric answers: what exactly are we checking and how do different reviewers count the same thing.

A practical set of metrics usually includes:

• accuracy: facts are correct, numbers and conditions not swapped
• completeness: the answer covers all parts of the question
• usefulness: concrete steps and format, not vague wording
• tone compliance: neutral and role-appropriate without extra promises
• verifiability: reliance on the correct source, no substitution or fabricated citations

Separate verifiability. If an employee asks “Which documents are needed to procure equipment under our rules?”, the model should rely on the internal procurement regulation, not “common practice.” If the source is missing or incorrect, a polished text often should not be released to work.

A separate block is safety. Here you evaluate risks: prohibited advice (e.g., bypassing checks), personal data leaks, disclosure of internal details, and confident answers where the correct move is “I don’t know” and to ask for clarification.

For partially correct answers a simple 0–2 scale is useful:

• 0: incorrect or dangerous
• 1: partially correct, but missing parts or lacking source confirmation
• 2: correct, complete, verifiable and safe

Expert review is needed where there are many nuances: legal phrasing, financial control, medical data, complex procurements. For typical FAQs, email templates, document search and basic reference often check out with checklists if sources are connected and actually verified.

Running tests and recording results

To make runs fair, first “freeze” conditions: the same prompt, the same system rules, identical tools (e.g., knowledge-base access) and the same temperature. Otherwise you compare settings, not models.

Versioning is crucial. Record not only the model version, but also the data version (knowledge snapshot), the question set and the evaluator instructions. Even small instruction changes can shift scores more than a model swap.

Keep a single protocol per test case:

• question and context (user role, channel, language)
• model answer (verbatim, without edits)
• sources or quotes if required
• score and rationale
• comment: what is wrong and how to fix it

Offline and online runs

Do offline runs before release. This is a controlled check where all test-suite questions are asked the same way and results are compared to previous runs.

Online checks are needed after release: sample a small percentage of real dialogues and manually score them with the same scale. If the internal assistant helps procurement, record whether it cites the correct regulations and whether it invents delivery conditions.

Storage and month-to-month comparison

Store results in a single repository where you can filter by date, department, model version and data version. For monthly comparison keep a snapshot of the test suite and a summary: average score, share of critical errors and a list of recurring failures. This turns testing from a one-off action into a measurable process.

A/B when changing models: how to compare fairly

Fair A/B comparison is needed when you change not only the LLM but also how it operates in your environment. Otherwise you may credit the model for differences caused by prompt, knowledge base or delivery settings. If your goal is to measure quality, first fix everything else and compare one factor at a time.

What to freeze

Agree what counts as “variant A” and “variant B.” Commonly compared factors include model/provider, system prompt and response template, knowledge base and search rules, fragment ranking (what gets into context), and safety settings (filters, refusals, policy).

Use identical questions and identical context. Run the same test suite, don’t mix new tasks on the fly and don’t give one side more source data. Ideally distribute requests randomly so time of day, load and user mix don’t skew results. If humans score answers, do blind reviews: don't show which model produced the answer.

How to decide

Set acceptance not as “on average better,” but as thresholds. For example: accuracy no lower than N, dangerous advice share strictly below M, percentage of answers with correct source references at least K. It helps to inspect by segment: finance, procurement, HR, support. One model may excel in one department and lag in another, which gets lost in an overall average.

If one model is more accurate and another has a friendlier style, decide by risk. For regulatory tasks accuracy and verifiability matter more; presentation can be improved by prompt and templates. For people-facing reference answers, a clearer style may be acceptable — but only if minimum factual and safety thresholds are met.

Common mistakes and pitfalls in LLM evaluation

The most common mistake is confusing what you’re actually improving. The team may like that the model answers quickly and confidently, while users care more about accuracy and verifiability. If the pain is factual errors, metrics for speed and “text prettiness” only distract.

Another trap is leaking the correct answer into the reference. This happens when an evaluator has already seen “the right way,” or when the reference answer leaks into model prompts, instructions or examples. Then the model looks better than it is in real use, and quality drops after release.

Too-general questions also spoil the picture. Prompts like “tell me about procurement” or “explain company policy” are hard to check for facts, and evaluations become subjective. Ask questions with concrete checks: amount, deadline, exception, process step, or a document clause.

When the test suite stops being a test

If you continually tune prompts or models specifically to your question set, the suite loses independence. It’s like studying a closed list of exam questions: scores rise, but real help may not. Keep part of the suite “closed” (don’t use those cases in tuning) and regularly add new cases.

Confusion with sources and versions

A separate error class is mixing document versions in one run. For example, half the questions use an old regulation and half the new, and it’s unclear whether the model or the data is at fault. Simple rule: fix source version for each case (date, number, edition) and check the model used that version. If sources change often, run tests in batches by document snapshot instead of mixing everything.

Quick checklist before release or model change

Before rolling out a new model or changing settings, do a short but hard check. It helps catch the most expensive errors before users see them.

Check six things that usually make results drift:

• business coverage: the test suite includes questions from key departments (finance, procurement, HR, legal, support, sales)
• cost of error: critical topics are marked where a wrong answer leads to fines, missed deadlines, leaks, wrong purchases or bad client decisions
• verifiability: each question has a current source or is explicitly marked “no source” and requires cautious handling
• negative cases: traps and refusal checks are included (what the model does when there’s no data, the request is harmful, or the user asks to break rules)
• repeatability: settings are recorded (model version, prompt, temperature, system instructions, search mode, citation format)

After the run decide in advance what counts as “pass.” Set release thresholds (for example, critical topics must be error-free) and define rollback: who decides, how quickly the previous version is restored, and what incidents are stop signals.

A small orientation: in a multifunction company (like a manufacturer and integrator such as GSE.kz) you often see skewed performance: the model is great on IT but stumbles on procurement rules or financial phrasing. The checklist helps catch that before release.

Example: how a company evaluates an assistant for finance and procurement

Imagine an internal assistant for finance and procurement in a manufacturing company like GSE.kz: an employee asks a question in chat and the assistant replies according to approval, payment and procurement rules. The test goal is simple: answers must be accurate, verifiable and safe, without invented rules or off-the-cuff advice.

The test suite was built from real requests over 2–3 months and augmented with questions that commonly cause errors. The resulting set requires both factual correctness and careful wording.

Examples from the suite:

• What purchase limit applies without a tender and who approves it?
• What is the approval route for a contract with a new supplier?
• Give a template email to a supplier about moving a delivery date.
• When is payment expected for an invoice that has already been approved?
• What documents are needed to close an advance payment?

For each question the reference included a short correct answer and pointers to internal sources (procurement policy, payment regulation, approval matrix). The reference recorded not only what to say, but what not to say. For example: don’t promise payment timelines without checking status; don’t state limits without citing the document edition.

An A/B run was conducted on the same test suite and with the same settings (temperature, system instructions, knowledge base). The new model handled email templates better and cited the relevant regulation more often. But new errors appeared: it sometimes mixed up approver roles for non-standard amounts and confidently answered where it should have asked a clarification.

After the run the team:

• expanded the knowledge base with borderline cases and examples
• strengthened bans on promising payment dates and guessing limits
• added a rule: ask one clarifying question when context is unclear
• repeated A/B on the updated set and confirmed improvements

Thus tests became a way to quickly find what to fix: documents, model prompts or safety rules — not a one-off check.

Next steps: keeping quality over time

Answer quality evolves: documents change, new common questions appear, and people phrase things differently. Therefore checks should be a regular process with clear roles and a calendar.

Assign process and responsibility

A simple scheme works best: each department has an owner (a lead or methodologist), and the LLM team has a single coordinator. Set minimal rules to avoid endless debates:

• the department owner adds 5–15 new real questions to the suite monthly
• the coordinator runs a short check weekly and collects issues in one list
• sources (policies, regulations, price lists) are stored with version and date
• any change to prompt, search tool or model goes through the tests
• there is a single place showing history of results and reasons for drops

Automation helps: collect request logs, flag answers with complaints and regularly add such cases to the set so it reflects real work.

Roll out via pilot and quality gates

Quality is easier to maintain with clear thresholds and gradual expansion:

• pilot on 1–2 departments with a limited scope
• release quality threshold (e.g., accuracy on critical questions and no sourced errors)
• limited rollout to a subset of users and collect feedback
• expand to new departments only after two stable runs

If you also need AI infrastructure, source versioning and a reliable run/monitoring environment, this is often done with GSE.kz (gse.kz) as manufacturer and systems integrator providing 24/7 technical support. The critical thing is that the technical layer supports update discipline and checks, not replaces them.