6 Sept 2026•4 min read
You cannot filter your way out of untrusted text reaching a model. What you can do is make sure that text has no authority when it arrives.
27 August 2026•3 min read
Every few months a new list of prompt-injection defences circulates, mostly variations on detecting malicious instructions in retrieved content. They help at the margin and none of them are a solution, because the underlying problem is not detection. It is that we routinely hand a model untrusted input and privileged tools in the same breath, then ask it to keep them straight.
Treat model output as coming from whoever authored the least trusted input in the context. If your agent read a web page, then the agent's proposed actions are as trustworthy as that web page. Once you internalise this, the mitigations stop being about wording and start being about permissions.
The attack that gets underrated is not destructive, it is extractive: text that persuades the agent to include sensitive context in a URL, an image request, or an outbound message. Defences are unglamorous and effective. Do not let the model construct arbitrary outbound URLs. Strip or proxy images in rendered output. Allow-list domains for any network egress. Log every outbound call with its full payload.
Add an adversarial suite to your evaluation set with cases drawn from real attack patterns: instructions hidden in documents, in file names, in code comments, in HTML attributes, in the middle of otherwise legitimate content. Run it on every change. When one succeeds, the fix belongs in the architecture, not in the system prompt.
A system prompt is a suggestion. A permission check is a rule. Only one of them survives an attacker who is better at writing than you are.
There is no wording that makes a model reliably ignore instructions in its input, and designing as though there might be is the actual vulnerability. Assume injection succeeds, and make sure the successful attacker inherits an account with almost no power.
@umarrafique923
Author and writer at CandyWrite. Sharing knowledge, tutorials, and reflections on technology, design, and ideas.
Join 12,000+ readers getting our Saturday morning editorial dispatch with our top essays and reading recommendations.
6 Sept 2026•4 min read
3 Sept 2026•3 min read
5 Sept 2026•4 min read
8 Sept 2026•5 min read
Discussion (0)
Join the conversation. Sign in to leave a response or reply to comments.