AI application & agent security

The job to be done

Give an AI application a bounded task and verify that its model, credentials, tools and runtime controls preserve those boundaries.

AI application and agent security protects the system using a model: its data, instructions, retrieved documents, credentials and actions. Start with the trust boundary. A document may be legitimate evidence while containing instructions that the agent must never obey. Runtime filters inspect content; authorization decides which actions are permitted; evaluation tests the assembled system; inventory and model scanning examine what is deployed. These are complementary control purposes, so this first edition is a research map rather than a vendor ranking. For an educational Agentic SOC deployment, a useful success criterion is an investigation that produces evidence without gaining permission to isolate a host merely because an alert asked it to.

Inside the segment

These capabilities answer different questions. Use the distinction to define the work before assembling a shortlist.

How the work flows

1. Draw the action boundary

   List model providers, retrieved data, tools and credentials. Mark which inputs are untrusted and which actions need a separate human approval. Give the investigation worker a distinct, minimally privileged identity.

2. Build a synthetic evaluation set

   Pair harmless business questions with test documents containing conflicting instructions and synthetic secrets. Define success by observable data access and tool outcomes, not by the fluency of the answer.

3. Wire the control into the path

   Apply content inspection at the actual model and retrieval boundaries. Enforce tool authorization independently. Record behavior when a filter times out, a policy rejects a call or a response streams before inspection completes.

4. Verify evidence and useful work

   Capture policy decisions and permitted or denied actions without retaining unnecessary prompt content. Verify that a denied write made no downstream change and that legitimate read-only work still completes.

5. Retest each meaningful change

   Version the model, system instructions, tools, permissions and test set together. Re-run representative cases after any change and investigate both missed attacks and newly blocked legitimate requests.

The environment changes the question

Use these scenarios to adapt the evaluation to your organization. They describe operational concerns, not a determination of compliance.

What drives the operating cost

• Request volume, text or token size, model calls made by evaluators and retained telemetry can have separate billing meters.
• Inline checks add latency and availability dependencies; benchmark retries, streaming and outage behavior for the actual application.
• Self-hosted libraries shift configuration, model hosting, upgrades and operating responsibility to your team.
• Continuous evaluation requires maintained test cases, adjudication of ambiguous results and time from application owners.

Questions worth asking

1. Where is policy enforced when model output requests a real action?
2. Can a compromised document cause access beyond the assigned task?
3. Which protections are deterministic and which depend on model judgments?

Common assumptions to check