System prompt

The system prompt is where a feature's behavior lives. It tells the model what job it is doing, who the output is for, what it must and must not do, what format to return, and how to handle edge cases. Most provider APIs give it a dedicated field separate from the conversation messages.

In a product, the system prompt is code. It should be versioned, reviewed, and tested like any other part of the feature, because a one-line edit can change behavior across every user. Keep it specific to one feature rather than writing a single giant prompt for the whole app; a summarizer and a reply drafter need different instructions.

Two practical points shape how you write it. First, keep the stable parts at the start and put per-request details (the user's name, today's date, retrieved documents) later, so prompt caching can reuse the prefix. Second, never treat it as a security boundary. Users and documents can try to override it, and determined users can often extract it, so don't put secrets in it and enforce permissions in your code.