Prompt engineering that survives production
A production prompt has to work for every input your users send, not for the one you tried. Write it like a brief for a capable new colleague: the task, who it's for, the context they need, the exact output format, the constraints, what to do when the input is unclear or the answer isn't available, and an example or two of good output. Then treat it like code: keep it in version control, test it on a set of real inputs including the bad ones, and change it only when the tests show it got better.
Chat prompting and production prompting are different
In a chat, you see the answer immediately. If it's not good, you rephrase and ask again, and you are the one who decides when it's good enough.
In an application, nobody is there to rephrase. The same prompt runs thousands of times, with inputs you never saw: long ones, empty ones, ones in other languages, ones that are angry, ones that try to break the system. The output goes into code, not to a person, so a slightly different format can break the next step. And a change that improves one case can quietly make ten others worse.
So a production prompt is less like a question and more like a specification.
Parts of a good prompt
Task
Say exactly what the model should do, with a verb and an object. "Classify this support email into one of four categories" is a task. "Help with emails" is not.
Audience and role
Say who the output is for and, if it helps, the perspective the model should take. "Explain this to a high school student using one real-world example" produces a very different answer from "explain this to a senior engineer". Role lines like "act as a senior engineer" can help set the level and vocabulary, but they are not magic, so pair them with concrete instructions.
Context
Give the information the model needs and can't know: your product, your policies, your users, the documents for this question. Most poor answers in applications come from missing context, not from poor wording.
Format
Say exactly what the output should look like. For people: length, structure, tone, reading level. For code: a JSON schema, and better, structured output enforced by the API.
Constraints
Say what the model must not do or must stay within: word limits, topics to avoid, sources it may use, claims it must not make. State constraints positively where you can ("answer only from the documents below") rather than only as prohibitions.
Edge cases
This is the part most prompts miss. Say what to do when the input is empty, off-topic, in another language, ambiguous, or asks for something the model shouldn't do, and what to do when the answer isn't in the provided information. A model that knows how to say "I can't answer this from the documents provided" hallucinates much less than one that was only told to answer.
Examples
One to three examples of good input and output often fix formatting and tone problems faster than any description. See few-shot examples.
A vague prompt and a production prompt
A vague prompt:
Summarise this customer email.
A production version of the same task:
You summarise customer support emails for the support team at an online electronics store.
Task: write a summary of the email below for an agent who has 10 seconds to read it.
Output: JSON with these fields:
- summary: one sentence, at most 25 words, in English, even if the email is in another language
- customer_goal: what the customer wants, in at most 10 words
- order_id: the order ID if one is mentioned, otherwise null
- urgency: "high" if the customer mentions a safety issue, a legal threat, or a deadline within 48 hours, otherwise "normal"
Rules:
- Use only information in the email. Do not guess an order ID.
- If the email is empty or not a customer message (for example spam), set summary to "not a customer message" and the other fields to null.
Email:
<email>
{email_text}
</email>
The second prompt is longer, and every line in it exists because of a failure you would otherwise see in production: summaries too long to read, invented order IDs, unclear urgency, spam treated as a customer message.
Separate instructions from data
Put user input and documents inside clear delimiters, like XML-style tags (<email>, <documents>), and say in the instructions that the content inside them is data to work on, not instructions to follow.
This makes the prompt easier for the model to follow, easier for you to debug, and reduces the risk of prompt injection, where text inside a document tries to give the model new instructions.
Ask for reasoning when the task needs it
For tasks with several steps, like a calculation or a decision with multiple rules, asking the model to work through the steps before the final answer improves accuracy. This is chain-of-thought prompting. In production, separate the reasoning from the answer, for example in different JSON fields or tags, so your code reads only the answer. For simple tasks like classification or extraction, skip it, because it adds tokens and latency without helping. Reasoning models do this on their own.
Techniques with names
You will see these terms often, and they are simpler than they sound.
- Zero-shot: no examples, the model relies on what it learned in training. "Classify this text as positive or negative."
- Few-shot: a few labelled examples in the prompt, so the model sees what you mean.
- Chain of thought: the model works step by step before answering.
- Tree of thoughts: the model explores several approaches, compares them and picks one, useful for open-ended decisions with more than one reasonable path.
- Prompt chaining: break a big task into several smaller prompts, each with one job, where the output of one is the input of the next.
- Self-check: a second step asks the model (or another model) to check the first output against the instructions or the sources.
Write for the model you use
Prompts don't transfer perfectly between models. Models are trained on different data and tuned differently, so a prompt tuned for one model can behave differently on another, and each provider publishes its own prompting guidance. When you switch models, run your tests again and expect to adjust.
Common mistakes
- Testing on two or three hand-picked inputs.
- Fixing one bad output by adding a rule, without checking what the rule does to everything else.
- Stuffing too much into one prompt, so the important instructions get lost. If a prompt does five jobs, split it.
- Contradictory instructions written at different times by different people.
- Using a prompt to do something code should do, like counting, exact formatting of numbers or checking that a value is in a list.
- Leaving the prompt as a long string in the middle of the code, where nobody reviews changes to it.
A checklist before you ship a prompt
- Is the task one clear job?
- Does it say who the output is for and what format it needs?
- Does it include the context the model can't know?
- Does it say what to do when the input is bad or the answer isn't available?
- Are inputs and documents inside clear delimiters?
- Is the output validated by code?
- Has it been tested on at least 20 to 30 real inputs, including bad ones?
- Is it in version control, with the test results for this version?
From the floor
In the cohort, I showed the difference with one example: "tell me about AI" gave a vague, general answer, and "in three sentences, explain the main application of AI for small businesses" gave an answer about automation, customer support, marketing and bookkeeping, because the prompt now said who it was for and how long it should be. Changing "small businesses" to "marketing" changed the vocabulary of the whole answer.
One builder asked whether the same prompt would give similar answers on ChatGPT, DeepSeek and Gemini. They won't always, because each model is trained on different data and tuned differently, so if you move a prompt to another model, you test it again.
A technique I use myself when I'm learning something new is to ask the model to explain it simply and then give me the subtopics to learn next, so the answer becomes a plan, not only an explanation.
Related
- System prompts
- Few-shot examples
- Prompt testing
- Pydantic and structured data