Shape Chat Model Behavior in the Chat Playground

Lab overview

Turn a general-purpose model into Contoso's IT help desk assistant. In this lab, you get a baseline response from contoso-chat, give it a set of instructions and test how they change its behavior, experiment with the token limit and reasoning effort, and view the code an application would use to call it. You'll also see the one thing instructions can't do: give the model knowledge it doesn't have.

Why you'd do this

A freshly deployed model will answer anything, in any tone, as if it works for nobody in particular. The instructions (what the API calls the system prompt) are how you fix that without training anything: a few paragraphs telling the model who it is, what it covers, what it refuses, and how to format answers. In this lab the same password-reset question gets a generic answer before the instructions and a Contoso-specific, correctly formatted one after. The chili recipe prompt proves the scoping works the other way too. This is the cheapest, fastest lever you have for model behavior, and it's the first thing to adjust when an assistant is being too chatty, too vague, or wandering off topic.

The password expiration policy question is the point of the lab. The model answers confidently and wrong, or admits it doesn't know, because instructions can change behavior but can't add facts. That gap is what grounding, the subject of the next labs, is designed to close. Knowing where instructions stop is what keeps you from trying to stuff a policy manual into a system prompt.

The two parameters matter for cost and user experience. Max completion tokens is a hard ceiling on the length of an answer; set it too low and responses cut off mid-sentence, set it high and a runaway answer can get expensive. Reasoning effort controls how much the model thinks before it responds: low is fast and cheap and fine for lookups, high takes longer, costs more tokens, and produces more thorough diagnosis, which the VPN troubleshooting prompt makes visible. You'll pick one of these per deployment in a real app, and the playground is where you find out which.

The Call model tab closes the loop: everything you tuned here is reproduced in code by pointing at the endpoint and the deployment name contoso-chat. For certification prep, this maps to the AI-901 objectives on AI agents and their components and the AI-103 material on agent instructions in Foundry Agent Service.

Lab details

• Deployment: contoso-chat
• Instructions: Contoso IT help desk assistant system prompt
• Test Prompts: password reset, chili recipe, password expiration policy, MFA setup, VPN troubleshooting
• Max Completion Tokens: 16,384 default, tested at 100
• Reasoning Effort: low compared with high

Learning objectives

1. Write instructions that scope a model to a specific role and set of topics.
2. Show how a token limit affects a model's response.
3. Compare model output at different reasoning effort levels.
4. Locate the endpoint and deployment name an application uses to call the model.

Exam relevance

• AI-901 — Microsoft Azure AI Fundamentals — AI agents and their components, grounding with your own data.
• AI-103 — Developing AI Apps and Agents on Azure — create a prompt agent in Foundry Agent Service, agent instructions, tools, and versions.

How this lab fits into the project

This is lab 3 of 12 in the Contoso Knowledge Assistant project. Contoso's IT help desk fields the same policy questions all day, so over this series you'll build an AI agent that answers them from the company's own documents, then ground it, secure it, evaluate it, monitor it, and publish it to Teams. This lab writes the help desk instructions that become the agent's personality, and shows the gap that grounding will close.

Step-by-step instructions

1. Open the playground:
   • In the Foundry portal, select Build, then click Models.
   • Click contoso-chat to open its playground.
2. Get a baseline with the default instructions:
   • Leave the Instructions field at its default value.
   • In the chat field, type How do I reset my password? and click the send button.
   • Note that the answer is generic: the model doesn't know who it works for or what system you mean.
3. Add a system prompt:
   • Click the Online Notepad browser tab, press Ctrl+A to select the custom instructions, and press Ctrl+C.
   • Click the Microsoft Foundry browser tab.
   • In the Instructions box, double-click the default text and press Ctrl+V to paste the custom instructions.
   • Click the new chat icon at the top right of the chat pane.
   • Type How do I reset my password?, click send, and note the change in tone, length, and format from the baseline.
   • Type What's a good chili recipe?, click send, and note the response.
   • Type What is Contoso's password expiration policy?, click send, and note that the model doesn't know.
4. Test the output token limit:
   • Click the Parameters icon to the right of the model selector.
   • Note the Max Completion Tokens default of 16384 and Reasoning Effort set to low.
   • Double-click 16384 and type 100.
   • Click the new chat icon, type Explain step by step how to setup multifactor authentication on a new phone., and click send.
   • Note how the answer cuts off when the response reaches the token limit.
   • Click New chat in the banner.
   • Click the Parameters icon, double-click 100, and set it back to 16384.
5. Compare reasoning effort:
   • Click in the chat box, type A user can sign into email but not the VPN, and their password was changed yesterday. List the likely causes in order., and click send.
   • Note the response time and token count shown under the answer.
   • Click the Parameters icon, open the Reasoning Effort dropdown, and click high.
   • Click the new chat icon, type the same prompt, and click send.
   • Compare the response time, token count, and depth of the answer to the first run.
   • Click the Parameters icon, open the Reasoning Effort dropdown, and click low.
6. View the code:
   • Click the Call model tab next to the Chat tab.
   • Note the endpoint URL and that the deployment name is contoso-chat.
   • Click the Chat tab to return to the conversation.