Build AI Agents With Foundry Agent Service: Tools, Functions and Memory (AI-103)

An agent is a model with instructions, tools and state that can decide what to do next: search knowledge, call an API, run code, then answer. Foundry Agent Service hosts agents for you: it stores their definitions and versions, runs tool calls, keeps conversations and emits traces. This lesson covers building one in Python.

Design the agent first

Before writing code, define:

• Role and goal: "You are Contoso's order support agent. Help customers track orders and request returns."
• Scope and constraints: what it must not do ("never promise refunds; never discuss other customers")
• Conversation tracking: one conversation per user session; long-term memory only if needed and allowed
• Tools: what it needs to read (knowledge, orders) and do (create a return), each with a clear schema
• Success criteria: what a good answer looks like, so you can evaluate it

Narrow agents with few, well-described tools are more reliable than "do everything" agents.

Tool types

Create an agent with a function tool

import json, os
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, FunctionTool

project = AIProjectClient(endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=DefaultAzureCredential())

get_order_status = FunctionTool(
    name="get_order_status",
    description="Look up the shipping status of a customer's order by order number.",
    parameters={
        "type": "object",
        "properties": {"order_id": {"type": "string", "description": "Order number, e.g. CT-10293"}},
        "required": ["order_id"],
        "additionalProperties": False,
    },
    strict=True,
)

agent = project.agents.create_version(
    agent_name="order-support",
    definition=PromptAgentDefinition(
        model="chat-main",
        instructions="You are Contoso's order support agent. Use tools to check orders. Never invent order data.",
        tools=[get_order_status],
    ),
)

create_version creates the agent, or a new version if it exists. Versions let you test changes before switching users over.

Handle the function call loop

With a function tool, the model asks and your code runs the function:

openai = project.get_openai_client()
conversation = openai.conversations.create()
agent_ref = {"agent": {"name": agent.name, "type": "agent_reference"}}

def lookup(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped", "eta": "2026-11-15"}  # call your real system here

response = openai.responses.create(
    conversation=conversation.id, input="Where is order CT-10293?", extra_body=agent_ref
)

calls = [item for item in response.output if item.type == "function_call"]
if calls:
    outputs = [
        {"type": "function_call_output", "call_id": c.call_id, "output": json.dumps(lookup(**json.loads(c.arguments)))}
        for c in calls
    ]
    response = openai.responses.create(conversation=conversation.id, input=outputs, extra_body=agent_ref)

print(response.output_text)

The pattern: model returns function_call (name + JSON arguments) → your code executes → send function_call_output with the same call_id → model writes the answer. Built-in tools like file search and code interpreter run inside the service, so there's no loop for those.

SDK shapes change between versions (for example, how you reference the agent), so check the current samples. The pattern is what's tested.

Add knowledge and other tools

Add tools to the same tools list, for example an Azure AI Search tool pointing at an index through a project connection, a code interpreter, or an MCP tool with a server URL, an allowed-tools list and approval settings. Each extra tool is another thing the model can misuse, so add only what the role needs.

Conversation and memory

• A conversation stores the turns of one thread on the service. Reuse its ID for follow-up questions.
• Long-term memory (a managed memory store in Foundry Agent Service, or your own database) holds user preferences and facts across conversations. Store only what's needed, tell users, and let them delete it.
• Long conversations consume context; the service truncates old turns, so put critical rules in the instructions, not early messages.

Make tools model-friendly

• Names that say what they do (get_order_status, not func1)
• Descriptions that say when to use them
• Parameter schemas with types, enums, formats and required fields
• Errors returned as readable output ("order not found") rather than exceptions
• Idempotent actions, with approval for anything costly or irreversible

Key takeaways

• Agent = model + instructions + tools + conversation state (+ memory), hosted and versioned by Foundry Agent Service.
• Tools: file search, AI Search/Foundry IQ, code interpreter, web, function, OpenAPI, MCP.
• Function tools: model emits function_call → your code runs it → return function_call_output with the call_id.
• Good tool names, descriptions and schemas are the biggest lever on agent reliability.

Next lesson: multi-agent workflows and approvals.