When we give an AI assistant a task, we want to know what else is shaping its answer. Our prompt is only part of the input. The application can add instructions about its role, tools, response format, and permitted behavior, some of which we won’t see in the chat window.
That lack of visibility deserves scrutiny, especially when we’re relying on the assistant for research or letting it act. It doesn’t establish a concealed political agenda. It does make us reluctant to treat a polished answer as the whole story.
What are hidden AI instructions?
By hidden instructions, we mean instructions you don’t see in the normal conversation interface. A system prompt is one way to supply them. An application may also add developer instructions, tool guidance, or context assembled for a particular task.
Some providers publish parts of this material. Anthropic publishes core system prompts for its web and mobile products, while noting that those updates do not apply to its API. OpenAI’s public Model Spec describes intended behavior and an instruction hierarchy. A behavior specification and a product’s exact runtime prompt are different forms of disclosure.
Neither source should be treated as an inventory of every instruction in every application built on that provider’s models. A third-party product can introduce its own context and constraints.
A system prompt is only one influence
It helps to separate four questions when evaluating an assistant:
- Training: What behavior has the model learned before the conversation begins?
- Runtime instructions: What does the provider or application ask it to do in this session?
- Retrieved context: Which documents, search results, memories, or tool responses influence the answer?
- Application controls: What does surrounding software block, transform, or permit?
We’d check each layer before attributing a refusal to one cause. The response alone may not tell us which layer was responsible.
Removing a visible prompt does not establish that the model has no remaining constraints. Equally, a surprising answer does not establish that a hidden instruction caused it. You need a controlled comparison to narrow that claim.
What a leaked prompt can establish
An authenticated prompt can show what a particular application instructed a model to do at a particular time. It can help explain intended behavior and suggest tests worth running.
An unattributed screenshot establishes much less. It may show a different product version, an incomplete conversation, an edited transcript, or text generated by the model rather than recovered from the application.
When reviewing a claimed disclosure, ask where the text came from, whether the provider acknowledged its authenticity, which version it applies to, and whether another person can reproduce the observation. Preserve the difference between an authenticated artifact and an interesting hypothesis.
We wouldn’t settle this by asking the assistant to explain itself. That explanation is another model output to check, not an independent account of how the application works.
Refusal, bias, and censorship need separate tests
A refusal tells you that the system declined a request. It does not automatically tell you why. It could be an appropriate boundary, an unnecessary refusal of a legitimate question, or a product restriction relevant to your work.
Over-refusal is a measurable problem. The XSTest research uses safe prompts and unsafe contrasts to examine exaggerated safety behavior. Its approach is useful because a model that answers everything and a model that refuses everything both have shortcomings.
Selective omission deserves attention too. An answer can appear helpful while leaving out facts that would change a reader’s interpretation. To investigate that, establish a reference set of relevant facts and apply the same completeness standard across comparable topics.
We’re cautious about calling an unexplained refusal censorship. A claim about systematic suppression needs a broader pattern and a clear definition. We’d apply that standard to every provider, regardless of country, ownership, or marketing position. A provider we prefer should face the same questions as one we distrust.
How to evaluate the assistant you actually use
Start with legitimate tasks from your workflow. Save the exact prompts, model identifiers, dates, available settings, and whether each interaction used a consumer app or an API. Record visible application instructions and mark unavailable ones as unknown.
Write the expected outcome before comparing providers. For a factual question, specify the facts a useful response should include and the sources that support them. For an authorized technical task, specify the permitted actions and evidence of completion.
Repeat a selection of prompts with neutral paraphrases. If the outcome changes, preserve both results. Do not report only the version that supports your starting belief. Have a person review disputed cases, particularly when evaluating viewpoint differences.
This work complements the action controls needed for code execution with MCP. A reasonable answer policy cannot replace a permissions system. Balance and factual quality need their own evaluation alongside access controls.
Make trust specific and revisable
We’d be comfortable giving a model a narrow job only after evaluating that job and deciding how to catch changes. Summarizing internal documentation and resolving a disputed factual claim call for different evidence and review.
We want more transparency from providers. Until we can inspect the relevant instructions and test the behavior, we’ll keep that uncertainty in the decision. Neither a hidden prompt nor a published one settles whether an application is reliable enough for the work we’re asking it to do.
