Prompt injection is becoming a trust-boundary engineering problem

Prompt injection is often demonstrated as malicious text that tells a model to ignore prior instructions. Production risk requires more: an attacker-controlled source, an interpreter that treats the content as relevant and a sink where the resulting decision matters. The sink may be a payment API, an email recipient, a repository write, a browser navigation, a secret-bearing tool or the answer delivered to a user.

This framing separates model influence from security consequence. Whether an attacker can change model behavior is probabilistic and model-dependent. Whether that influence can cross an authorization or confidentiality boundary can be constrained with conventional controls. A manipulated model need not become a successful incident if it cannot acquire credentials, reach sensitive data, call arbitrary destinations or modify high-impact resources.

Indirect injection expands the source set beyond the current user. A malicious webpage can target a browser agent, a document can target retrieval, a ticket can target a coding assistant, and a tool response can target the next call. Memory and multi-agent messages can persist or propagate the influence.

System messages can tell a model that retrieved documents are data, not instructions. Training can improve that distinction. Structural techniques can delimit or transform untrusted text. These are valuable layers, but the resulting behavior remains statistical.

Authority should be represented outside the prompt. An authenticated user or service defines a bounded task. A policy layer records which action classes, resources, destinations and fields are permitted. The model proposes a plan inside that envelope. The tool gateway independently checks whether the exact call fits the grant.

This separation makes failures diagnosable. A model failure is an unsafe proposal. A policy failure is a prohibited proposal reaching execution. A confirmation failure is misleading approval. A sandbox failure is execution escaping its boundary. Each domain can be tested and owned separately.

Applications often know that an email came from outside the organization or that a webpage is untrusted, then lose that fact when text is chunked, embedded, retrieved and concatenated into one prompt. A secure context pipeline carries origin, tenant, integrity, confidentiality, purpose and content digest alongside each chunk.

Provenance affects authority and disclosure. An unsigned public webpage cannot create authority to change payroll. A private customer record should not be sent to a URL extracted from that webpage. An approved tool may be authentic while returning user-controlled text.

Labels need enforceable meaning. Public-untrusted content may support analysis but may not determine recipients or executable commands. Confidential content may be usable by an internal summarizer but barred from external network sinks.

Prompt-injection detectors can find known phrases, obfuscation or semantic conflict. Model critics can compare a plan with the original task. Spotlighting and data marking can help distinguish external content. These mechanisms reduce attack success, especially against common attacks.

None proves that an action is authorized. Benign documents contain imperative language. Malicious instructions can be encoded, translated, split across sources or adapted after the attacker learns the defense. A clean classifier result is one input to risk assessment, not permission to attach credentials.

Operationally, detector signals should change routing. A high-risk result can remove write tools, require stronger confirmation, switch to isolation or send the task for review. Low-risk results still pass argument validation and policy.

Information-flow control asks whether data of a particular class may influence a destination. A public webpage can inform a comparison but may not determine where internal records are uploaded. A customer email can define a reply recipient while being barred from authorizing a large refund.

Exact token-level causality is unavailable, so practical systems use conservative taint. If untrusted content was present in the context used to produce an argument, the argument inherits an untrusted-influence label unless a separately defined transformation removes it.

Conservative propagation can create false blocks, so narrow tools matter. Instead of arbitrary HTTP, expose a bounded operation with server-selected destinations. Instead of shell access, expose a read-only parser. Narrow sinks make policy useful.

A function schema validates shape, not business authority. A valid amount can target the wrong account. A syntactically valid URL can be an exfiltration endpoint. Gateways canonicalize identifiers, resolve current state, enforce allowed operations and fields, attach a narrow credential and create idempotency before execution.

Tool names, descriptions and MCP metadata can contain untrusted content. A registry should allow qualified servers and pinned versions. Newly discovered tools can be restricted to read-only or denied pending review.

Confirmation should be generated from validated structured arguments and authoritative resource data, not free-form model text. The user sees the action, before-and-after values, recipient, sensitive data, irreversibility and policy exception. Approval binds to a digest and expires quickly.

A sandbox turns model compromise into a bounded incident. Code execution should have an ephemeral filesystem, no inherited host credentials, constrained CPU and memory, deadlines, syscall restrictions and explicit network policy. Browser agents should separate origins and profiles. MCP servers should use dedicated identities.

Network egress is central because prompt injection often aims to disclose data. An agent that can read private content and navigate to any URL has a simple exfiltration path. Allow destination classes, proxy requests, remove sensitive fields, cap payloads and require confirmation for new recipients.

Containment should limit persistence. Do not automatically write model conclusions or external instructions into long-term memory. Separate scratch state from reviewed durable memory. Expire credentials and destroy sandboxes after use.

A benchmark that asks whether a model repeats a forbidden sentence misses operational risk. The test should begin with an authorized task, place attacker content in a realistic source, provide actual or simulated tools and observe whether a prohibited sink is reached.

Attack sets should include direct and indirect instructions, obfuscation, multilingual content, images, HTML comments, delayed triggers, memory poisoning, tool-output injection and multi-agent propagation. Attackers should adapt after observing refusals. Benign tasks should include imperative text so the system cannot succeed by blocking every instruction-shaped sentence.

Production telemetry should surface unusual tool sequences, new destinations, policy denials, secret-like outputs, repeated confirmation and divergence from the original task. Every incident should improve detector coverage, tool scope, policy, confirmation and regression tests—not only add a phrase blacklist.