An uploaded document tells our AI to ignore its instructions. What should happen?
Untrusted content attempting instruction overrideTreat retrieved documents and user uploads as data, not as trusted instructions. Keep permission checks and action limits outside the model, and test that malicious embedded text cannot authorize tool use or reveal restricted information. Route suspicious behavior to the security owner.
What to do now
- Separate trusted instructions from document content
- Enforce authorization at every tool boundary
- Test adversarial document examples safely
- Record and remediate failures before broader use
When to escalate
Untrusted content can trigger unauthorized actions or disclosure.
The AI service owner and relevant authorized reviewer
Example
A document's instruction to export all files is ignored and cannot bypass the tool's access checks.
Mistake to avoid
Assuming a warning in the prompt replaces server-side access controls.
Prevent a repeat
Re-evaluate after changes to models, prompts, tools or source data; keep important decisions accountable to people.
Suggested ApprovedFlow setup
- Tasks
- Documents
- Approvals
Track evaluation and review tasks, and keep approved evidence with the work; do not claim the platform provides an unverified AI agent or automated diagnosis. See the platform
Related cases
- A colleague pasted confidential project documents into an unapproved AI service. What now?Unapproved information disclosure
- The AI search retrieves an obsolete policy. How can we prevent this?Stale knowledge retrieval
- Team members get conflicting instructions from their line managers. What should happen?Matrix authority conflict
- A senior stakeholder bypasses the project manager and assigns work directly. What should happen?Uncontrolled instruction channel