Choosing and Deploying AI Models in Microsoft Foundry: Types and Parameters (AI-901)

Microsoft Foundry gives you access to a large catalog of models from Microsoft, OpenAI, Meta, Mistral, DeepSeek, xAI and others. AI-901 tests whether you can choose a sensible model for a job, pick a deployment type, and set the parameters that shape its output.

Choosing a model by capability

Start from what the task needs, not from the most famous model.

You need to…	Choose
Chat, write, summarise, answer questions	A chat / language model (for example a GPT model)
Solve multi-step maths, logic or planning problems	A reasoning model
Understand images or audio in prompts	A multimodal model that supports that input
Create images	An image-generation model (for example gpt-image-1)
Power semantic search or RAG	An embedding model (for example text-embedding-3-small)
Run cheaply, fast, or on a device	A small language model such as Phi
Transcribe or speak	Speech models or Azure Speech

Then compare candidates on:

Quality for your task. The model catalog shows benchmarks and leaderboards, but test with your own prompts.
Cost per million input and output tokens.
Latency and throughput: how fast the first token arrives and how many tokens per second.
Context window: can it fit your documents?
Region and deployment availability.
Licensing and data rules, such as whether a model can be used commercially.

A good habit: start with a capable general model to prove the idea, then try smaller, cheaper models and keep the smallest one that meets your quality bar.

Deployment types

To use a model from code you usually deploy it in your Foundry resource and give the deployment a name. The deployment type decides where requests are processed and how you pay.

Standard (pay per token)

Global Standard: traffic can be processed in any Azure region worldwide. Usually the highest default quota and the first to get new models. A good default.
Data Zone Standard: processing stays within a geographic data zone, such as the EU or the US, while still balancing across regions inside it.
Standard (regional): processing stays in the region of your resource.

Provisioned (reserved capacity)

Global, Data Zone or Regional Provisioned: you reserve capacity in provisioned throughput units (PTUs) for predictable latency and throughput. Best for steady, high-volume production workloads. You pay for the reserved capacity, whether you use it or not.

Batch

Global or Data Zone Batch: you submit a file of many requests and get results back asynchronously, typically within 24 hours, at a lower price. Ideal for large offline jobs such as classifying thousands of documents.

Other options

Developer deployments are for testing fine-tuned models cheaply.
Some partner and open models are offered as serverless (pay-per-use API) deployments or on managed compute, where you pay for the virtual machines that host the model.

Microsoft adds and renames deployment options from time to time, so check the current list in the documentation, but the ideas stay the same.

Picking a deployment type: quick rules

Requirement	Deployment
Just getting started, variable traffic	Global Standard
Data must be processed in the EU (or US)	Data Zone Standard or Provisioned
Data must stay in one region	Standard (regional) or Regional Provisioned
Guaranteed throughput and steady latency	Provisioned
Huge job, not urgent, lowest cost	Batch

Configuration parameters

These parameters control how a model generates output. You can try them in the Foundry playground before using them in code.

Temperature (often 0–2): randomness. Low values (0–0.3) give focused, repeatable answers for facts, extraction and code. Higher values (0.8+) give more varied, creative output.
Top_p (nucleus sampling, 0–1): the model only considers the most likely tokens whose probabilities add up to top_p. Lower means more focused. Adjust temperature or top_p, not usually both.
Max output tokens: the upper limit on the answer's length. Too low and answers get cut off.
Stop sequences: text that tells the model to stop generating.
Frequency and presence penalties: discourage repeating the same words or topics.
System prompt / instructions: not a number, but the most powerful setting of all (see the prompt engineering lesson).

Reasoning models can work differently. Some don't support temperature and instead offer a reasoning effort setting (for example low, medium, high) that trades speed and cost for deeper thinking.

Example

response = openai.responses.create(
    model="my-gpt-deployment",   # the deployment name
    instructions="You extract facts. Answer in one sentence.",
    input="When was the Eiffel Tower completed?",
    temperature=0.2,
    max_output_tokens=100,
)

Quotas and limits

Each subscription has quotas per model and region, measured in tokens per minute (TPM) and requests per minute. If you exceed them you'll get rate-limit errors (HTTP 429). You can raise the TPM assigned to a deployment within your quota, request more, or use provisioned capacity.

Key takeaways

Choose models by capability first, then quality, cost, latency and context window.
Standard = pay per token. Global / Data Zone / Regional = where data is processed. Provisioned = reserved throughput. Batch = cheap and asynchronous.
Temperature and top_p control randomness; max tokens controls length.
In code you call a model by its deployment name.

Next lesson: the common AI workloads.