System Prompts vs User Prompts: Prompt Engineering for AI-901

The same model can give a vague, wrong answer or an excellent one depending on the prompt. Prompt engineering is the skill of writing instructions that get reliable results, and AI-901 asks you to create effective system and user prompts.

System prompt vs user prompt

In the Responses API, the system prompt is usually passed as instructions (or a message with the system or developer role), and the user prompt as input.

response = openai.responses.create(
    model="gpt-5-mini",
    instructions="You are a friendly travel assistant for Contoso Travel. "
                 "Only answer questions about travel. Keep answers under 100 words.",
    input="What should I pack for Dubai in July?",
)

A template for strong system prompts

Good system prompts usually cover five things:

1. Role: who the assistant is. "You are a support assistant for Contoso's billing team."
2. Task and scope: what it should and shouldn't do. "Help customers understand invoices. Don't give legal or tax advice."
3. Grounding rules: where answers must come from. "Answer only from the provided documents. If the answer isn't there, say you don't know."
4. Tone and style: "Be concise and polite. Use simple English."
5. Output format: "Reply in Markdown with a short answer, then bullet points."

Add safety rules too, such as refusing harmful requests and never revealing the system prompt.

Techniques for user prompts

Be specific. Compare:

• Weak: "Write about our product."
• Strong: "Write a 3-sentence product description of our noise-cancelling headphones for a busy commuter. Mention the 30-hour battery."

Give context. Include the data the model needs: the email to reply to, the table to analyse, the audience.

Use clear delimiters. Separate instructions from content with headings, triple quotes or XML-like tags in code, so the model knows which part is data:

Summarize the review between the triple quotes in one sentence.
"""
The battery lasts all week but the strap broke after a month.
"""

Show examples (few-shot prompting). One or two examples of input and the output you want are often more effective than long explanations.

Classify the sentiment as Positive, Negative or Mixed.
Review: "Fast delivery, great price." → Positive
Review: "Arrived broken." → Negative
Review: "Lovely design, but too small." →

Without examples it's zero-shot; with one example it's one-shot; with several, few-shot.

Ask for a format. If code will read the answer, ask for JSON with named fields, or use the model's structured output feature to enforce a schema.

Return JSON with the fields "product", "issue" and "urgency" (low, medium, high).

Our JSON formatter is handy for checking that output.

Break big tasks into steps. "First list the key facts, then write the summary" improves complex answers. Reasoning models do much of this thinking internally, so for them keep prompts clear and direct rather than spelling out every step.

Grounding: the most important technique

To reduce hallucinations, ground the model in trusted information:

1. Retrieve relevant content (from search, a database or uploaded files).
2. Put it in the prompt.
3. Instruct the model to answer only from it and to cite sources.

This is retrieval-augmented generation (RAG). In Foundry agents, tools like file search and Azure AI Search do the retrieval for you.

Iterate in the playground

Prompt engineering is experimental. In the Foundry playground you can:

• Edit the instructions and resend the same question
• Change temperature and max tokens
• Add example messages
• Compare models side by side

Keep a small set of test questions, including tricky and off-topic ones, and check every prompt change against all of them. Foundry's evaluations automate this at scale.

Common mistakes

• Putting permanent rules in the user message instead of the system prompt
• Asking several unrelated questions at once
• Not saying what to do when the answer is unknown
• Asking for "short" or "detailed" without saying how short or how detailed
• Trusting the prompt alone for safety: content filters and access controls still matter

Key takeaways

• System prompt = role, rules, scope, format for the whole conversation. User prompt = the specific request.
• Be specific, give context, use delimiters, show few-shot examples and request a format.
• Ground answers in your data and tell the model what to do when it doesn't know.

Next lesson: deploying a model in the Foundry portal.