AI-103 courseLesson 2 of 18 · 14 min
Choose the Right Models and Foundry Services for AI Apps and Agents (AI-103)
XUtilHub Team ·
Exam skills covered
- Choose an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
- Choose the appropriate Foundry services for generative tasks, grounding, vector search, agent workflows, or multimodal processing
- Choose an appropriate method for retrieval and indexing
- Choose appropriate memory, tool, and knowledge integration services for agent solutions
Most AI-103 scenario questions start with a choice: which model, which service, which retrieval method. This lesson gives you a decision framework for the first skill group of the exam: choosing the appropriate Foundry services for generative AI and agents.
Choosing a model
Microsoft Foundry's model catalog includes models from OpenAI, Microsoft, Meta, Mistral, DeepSeek, xAI and others. Match the model type to the task:
| Need | Model type | Examples |
|---|---|---|
| General chat, writing, complex instructions | Large language model (LLM) | GPT-5 family, GPT-4.1 |
| Cheap, fast, narrow, high-volume tasks, or edge devices | Small language model (SLM) | Phi family, GPT-5-nano |
| Multi-step problems: maths, planning, code, analysis | Reasoning model | o-series, GPT-5 with reasoning effort |
| Text plus images or audio in the same request | Multimodal model | GPT-4o, GPT-5 |
| Writing and fixing code | Code model | Codex models |
| Vectors for search and similarity | Embedding model | text-embedding-3-small / -large |
| Images or video from prompts | Generation model | gpt-image-1, Sora |
Then weigh the trade-offs the exam loves: quality vs cost vs latency, context window size, regional availability, and whether the model supports the features you need (tools, structured outputs, vision input). The Foundry portal's model leaderboards and benchmarks help compare quality, cost and throughput.
Rule of thumb: start with the smallest model that passes your evaluations, and route only hard requests to bigger or reasoning models.
Foundry Tools vs prompting a model
Foundry Tools are the task-specific services formerly known as Azure AI services: Azure Language, Azure Translator, Azure Speech, Azure Vision and Content Understanding.
| Choose a Foundry Tool when… | Choose an LLM prompt when… |
|---|---|
| You need predictable, deterministic output and a fixed API | The task is open-ended or needs reasoning |
| Volume is high and cost per call must be low | Labels or formats change often |
| There's a prebuilt capability (PII redaction, speech to text, translation with glossaries) | You need nuance such as tone or domain-specific summaries |
Many solutions mix both: Speech transcribes a call, then an LLM summarises it.
Choosing services for each job
| Job | Service |
|---|---|
| Generate text, chat, call tools | Model deployments in Foundry, via the Responses API |
| Ground answers in your data | Azure AI Search (vector, hybrid, semantic) or Foundry IQ knowledge bases |
| Vector search | Azure AI Search vector indexes (or vector support in Cosmos DB / PostgreSQL when data already lives there) |
| Agents with tools and conversations | Foundry Agent Service |
| Multi-step and multi-agent orchestration | Foundry workflows or Microsoft Agent Framework |
| Documents, images, audio, video into structured data | Content Understanding |
| Safety | Azure AI Content Safety guardrails |
Choosing a retrieval and indexing method
- Keyword (BM25): exact terms, product codes, names
- Vector: meaning and paraphrases ("cancel my plan" matches "terminate subscription")
- Hybrid: keyword + vector merged, the usual default for RAG
- Semantic ranker: re-ranks the top results with a language model for better precision
- Agentic retrieval: the model breaks a complex question into sub-queries, runs them and merges the results. This is what Foundry IQ knowledge bases do for agents.
Indexing choices: push (your code uploads documents) or pull (an indexer reads from Blob Storage, SQL or Cosmos DB on a schedule), plus chunking and embedding at ingestion. We go deeper in the retrieval pipelines lesson.
Memory, tools and knowledge for agents
Agents need three kinds of integrations:
- Memory: a conversation keeps the history of one thread. Long-term memory stores facts and preferences across sessions. Foundry Agent Service has a managed memory feature, or you can keep your own store, such as Cosmos DB.
- Tools: built-in tools (file search, code interpreter, web search, Azure AI Search, image generation), function tools that run your code, OpenAPI tools for REST APIs, and MCP tools from Model Context Protocol servers
- Knowledge: file search over uploaded files, Azure AI Search indexes, Foundry IQ knowledge bases, or connected sources such as SharePoint
Exam tip: if the scenario says "remember across sessions", think long-term memory. If it says "call our order system", think function or OpenAPI tool. If it says "answer from our policies", think knowledge or search tool.
Key takeaways
- Pick models by task, quality, cost, latency and features. Use SLMs for narrow, high-volume tasks and reasoning models for hard multi-step problems.
- Use Foundry Tools for predictable prebuilt capabilities and LLM prompting for flexible tasks.
- Azure AI Search is the default retrieval layer; hybrid + semantic is the usual RAG choice.
- Agents combine memory (conversation and long-term), tools (built-in, function, OpenAPI, MCP) and knowledge.
Next lesson: set up Foundry, deployments and CI/CD.
Check your understanding
Exam-style questions on this lesson. Get them all right to mark the lesson complete.
Finished reading?