Evaluate Generative AI Apps: Groundedness, Relevance, Quality and Safety (AI-103)

"It looked fine when I tried it" isn't a test strategy. Generative AI output varies, so you evaluate apps on a dataset, with metrics, before and after every change. AI-103 tests this directly: evaluate models and apps, including detecting fabrications, relevance, quality and safety.

What to measure

Two families:

• AI-assisted evaluators use a judge model (LLM-as-a-judge) to score, usually 1–5 with a reason. They need a model deployment.
• Code-based (NLP) evaluators such as F1, BLEU and ROUGE compute scores mathematically against ground truth.

Safety evaluators use a Microsoft-hosted safety service, configured with your Foundry project.

Build an evaluation dataset

A dataset is a JSONL file of test cases:

{"query": "How many days of parental leave do I get?", "context": "...policy text...", "response": "...app answer...", "ground_truth": "16 weeks paid"}

Good datasets cover typical questions, edge cases, out-of-scope questions ("what's the CEO's salary?") and adversarial prompts. Start with 50–200 rows. You can generate synthetic questions from your documents, then review them, and add real production samples later.

Run evaluators in Python

from azure.ai.evaluation import GroundednessEvaluator, RelevanceEvaluator, evaluate

judge = {
    "azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
    "azure_deployment": "judge-model",
}

groundedness = GroundednessEvaluator(model_config=judge)
print(groundedness(
    query="How many days of parental leave do I get?",
    context="Employees receive 16 weeks of paid parental leave.",
    response="You get 20 weeks of paid leave.",
))  # low score + reason: 20 weeks isn't supported by the context

results = evaluate(
    data="eval.jsonl",
    evaluators={"groundedness": groundedness, "relevance": RelevanceEvaluator(model_config=judge)},
    azure_ai_project=os.environ["FOUNDRY_PROJECT_ENDPOINT"],  # log the run to Foundry
)
print(results["metrics"])

• One evaluator can score a single row interactively.
• evaluate() runs several evaluators over a dataset and returns aggregate metrics.
• Logging to the project shows the run in the Foundry portal, where you can compare runs side by side.

The azure-ai-evaluation package and Foundry's cloud evaluation API evolve quickly; check current samples for exact parameter names.

Evaluate in the portal

Foundry's Evaluation page runs the same evaluators without code: pick a model, agent or dataset, choose evaluators, and compare results. It's handy for model selection: run the same dataset against three deployments and compare quality, cost and latency.

When to evaluate

1. Model selection: compare candidate models on your data
2. Development: after each prompt, retrieval or tool change
3. CI/CD gate: fail the build if groundedness or safety drops below a threshold
4. Production: continuous evaluation on sampled traffic to detect drift, plus red teaming for safety

Reading results

• Low groundedness, good retrieval → the prompt lets the model improvise; tighten grounding instructions
• Low groundedness, poor retrieval → fix search first (chunking, hybrid, semantic ranker)
• Low relevance → the answer drifts; clarify instructions or route to a better model
• Low tool call accuracy → improve tool names, descriptions and parameter schemas

Key takeaways

• Groundedness detects fabrications; relevance checks the answer fits the question; coherence and fluency check quality; safety evaluators check harm; agent evaluators check intent, task adherence and tool calls.
• AI-assisted evaluators need a judge model deployment; NLP metrics need ground truth.
• Use evaluate() over a JSONL dataset, log runs to Foundry, and gate releases in CI.
• Diagnose by combining metrics: retrieval vs generation problems need different fixes.

Next lesson: build agents with Foundry Agent Service.