A course segment explaining how instruction hierarchy, direct and indirect prompt injection, and provenance-based defenses determine whether an AI agent follows authorized instructions or gets hijacked.
Adapted from @sairahul1# Become AI Agentic Engineer in 6 Weeks (Full Course) - Part 2
Your agent worked perfectly in testing.
Then someone pasted a webpage into it.
The agent read the page, found hidden instructions inside it, and started searching the user's private files.
That is not a hypothetical.
That is how real production agents get compromised.
This is Part 2 of the AI Agentic Engineer series.
In Part 1, we covered how LLMs actually work: tokens, inference, temperature, hallucination, RAG, and tools.
In Part 2, we go deeper into prompt engineering and context security, the things that determine whether your agent actually follows instructions in production or gets hijacked by untrusted content.
Save this. You will reference it every time you build an agent.
Quick recap of where we are
Part 1 established the foundation:
Part 2 builds on that.
Now that the agent can use tools and take real actions, the security model changes completely.
A chatbot being tricked → bad answer.
An agent being tricked → sends email, modifies code, reads private data, issues refund, deletes something.
The question shifts from "how do I stop the model saying something wrong" to "how do I design the system so that even if the model is manipulated, it cannot exceed the user's permissions."
Let's build that system.
## The instruction hierarchy
Before we get to attacks, you need to understand how the LLM processes instructions.
Not all instructions are equal.
They have a priority order:
This means:
Here is the critical mistake engineers make:
They treat instruction hierarchy as their security model.
It is not.
The LLM is still probabilistic. Hierarchy helps. It does not guarantee.
Using "system prompt says no" as your only security control is like protecting an admin API with a comment:
instead of actual authorization.
Direct prompt injection
The attacker uses the normal user input channel to override your application instructions.
The attacker is trying to manipulate the model through the user input channel.
Why can't the model just ignore this?
Because the LLM is probabilistic.
Repeated exposure to "ignore previous instructions" in training means those words carry real weight in the model's learned patterns.
The defense:
Authorization belongs outside the model.
If the model produces:
Your backend asks:
Then allows or denies.
Not: LLM decides authorization → execute.
This is exactly normal backend engineering. AI does not repeal it.
## Indirect prompt injection — the sneaky one
This is more dangerous than direct injection.
The user did not write the malicious instruction.
It came from content the agent retrieved.
Agent fetches the page and finds:
The attack path:
The user never saw the malicious instruction. They just asked for a summary.
This is indirect prompt injection.
Why it's so dangerous for agents:
Agents constantly consume external content.
Every piece of external content is a potential attack surface.
Imagine a coding agent reading a GitHub issue:
That second paragraph is content inside an issue.
Not an authorized development instruction.
The agent has to be designed to understand the difference.
## Provenance — who said this?
This is the core of the defense.
Suppose the model receives:
Without provenance, they are just three strings.
With provenance:
Now the architecture can reason about this properly.
How to implement provenance in practice:
Notice the label: [EXTERNAL - UNTRUSTED]
This signals to the model that this content is data, not commands.
Important caveat:
Labels help. They are not a complete security boundary.
You still need deterministic controls outside the LLM.
## The confused deputy — your agent shouldn't be a permission bypass
This concept comes from classical security.
Imagine:
Bad architecture:
Alice has no payroll permission.
But she just used the agent's credentials to read it.
The agent became a confused deputy — it has legitimate authority, but used it on behalf of someone who shouldn't have that authority.
The correct architecture:
The user's identity and permissions propagate through the entire system.
The agent never gets "god mode" credentials that anyone can invoke through it.
This is backend authorization that you already know.
AI does not change the principle.
## Least privilege — only give the agent what it needs
This is the most powerful single control.
If an agent cannot access sensitive data, a successful injection cannot exfiltrate it.
The dangerous capabilities exist and the LLM is your only gate.
The dangerous capabilities never enter this agent's available tool set.
Even if the model is fully compromised:
This is why tool selection is a security decision, not just a performance decision.
Read vs write capabilities:
Read-only agents have a dramatically smaller blast radius.
Always ask: does this agent actually need write access for this task?
## Tool argument validation — the step everyone skips
The model is allowed to call refund_order.
The user legitimately owns order 123.
Model produces:
Tool name: permitted ✓ User owns order: verified ✓ Amount: $500,000 for a $49 product
You need full validation before execution:
The full validation chain:
Every layer exists because the one before it is not sufficient alone.
## Data exfiltration — the attack nobody thinks about
Your research agent can:
An attacker puts this in a webpage it retrieves:
The attack chain:
The problem is not just "the model followed a bad instruction."
The problem is: the system allowed untrusted content to create a path from sensitive data to an external sink.
Sources and sinks — the mental model:
If your agent can access both sensitive sources AND powerful external sinks, think carefully about what untrusted content can connect them.
The fix:
Separate agents by capability.
A reading agent that cannot write externally cannot exfiltrate.
A writing agent that cannot access sensitive data cannot exfiltrate.
Combining both in one agent is where the risk lives.
## Human approval for high-impact actions
Not every action needs human approval. That defeats the purpose of an agent.
But some do.
The questions to ask for any action:
Irreversible + high-impact + external = require human approval.
Reversible + low-impact + internal = usually fine to auto-execute.
## Defense in depth — no single layer solves everything
The mistake: treating any one control as the complete solution.
A robust system combines all of them:
No single layer has to be perfect.
Each layer catches what the previous missed.
That is defense in depth.
One more thing: don't rely on blacklisting phrases.
You cannot reliably filter "ignore previous instructions."
Attackers rephrase. Legitimate documents discuss prompt injection.
The defense is architectural and deterministic, not a blacklist.
## The full production agent security architecture
Put it all together. This is what a serious agent system looks like:
Notice where authorization sits.
After the model proposes, before execution.
The model is inside the architecture. It does not own the architecture.
## Three interview-ready scenarios
These are the kind of questions you will face for AI Engineer roles.
Work through each one before reading the answer pattern.
Scenario 1 — Email agent indirect injection
You built an AI email assistant.
The user asks: "Summarize my latest emails and tell me which need action."
One email contains:
Questions:
1. What type of attack is this?
1. Why is "tell the model to ignore malicious instructions" insufficient as the only defense?
1. What architectural controls would you add?
Answer pattern:
Scenario 2 — Confused deputy in SaaS
Your SaaS has Alice (employee) and Bob (CFO).
The AI agent has database credentials that can read the full payroll table.
Alice asks: "What's Bob's salary?"
The LLM calls get_salary("Bob") and the DB returns it.
Answer pattern:
Scenario 3 — Dangerous tool combination
Your team argues: "Each tool is individually safe. Search is safe, reading docs is safe, email is normal."
The agent has: search_web() + read_internal_documents() + send_email()
What is wrong with this reasoning?
Answer pattern:
## Updated mental model after Part 2
Practice questions for Part 3
These are real interview questions. Work through them.
Q1 — Blast radius thinking
You have two agents:
Agent A: search_web() + read_public_webpages()
Agent B: read_customer_database() + send_arbitrary_http_requests() + send_email() + issue_refunds() + delete_accounts()
Both are successfully prompt-injected.
Compare the blast radius. What does this tell you about how to design agents?
Q2 — Production email assistant
You need to build an email assistant that:
- reads the user's inbox
- drafts replies for approval
- can send emails after explicit user confirmation
Design the security architecture. What tools does each component have? Where does human approval happen? What gets logged?
Q3 — RAG system with conflicting documents
Your retrieval system pulls two documents:
- Document A (version 1, 6 months ago): "Refund window: 30 days"
- Document B (version 2, current): "Refund window: 14 days"
Both are passed to the LLM.
What can go wrong? Who is responsible for preventing it? How do you fix this at the retrieval layer, not the LLM layer?
Q4 — Tool argument injection
The model has access to transfer_funds(from_account, to_account, amount).
A user legitimately authenticated asks to transfer $100 to their savings account.
But the model produces:
What validation layers would catch this? List all of them in order.
Part 3 is coming
Next: Tool Calling and Agent Architecture.
The core agent loop. ReAct pattern. Planner/executor. State machines. Workflows vs agents. Stopping conditions.
The section where you go from "understanding LLMs" to "building systems that actually work."
If this was useful:
→ Repost to share it with every developer building agents
→ Follow @sairahul1 for Part 3 and the rest of the series
→ Bookmark this — the validation chain and security architecture are the two things you reference every build
I write about AI, building products, and systems that work while you sleep.