How Can a Zero-Trust Approach Help Guard Against LLM prompt injection attacks?

As artificial intelligence becomes more embedded in business systems, it introduces a new class of cybersecurity risks. Large language models (LLMs), in particular, are capable of interpreting and acting on human-like instructions. While this makes them highly useful, it also exposes businesses to new vulnerabilities.

One major issue is that traditional security models, which rely on perimeter defenses and known signatures, are not designed to address the latest generation of threats, such as prompt injection attacks. To protect these tools, businesses need to move toward new, more adaptive security solutions. A key element of this will be using a zero-trust approach, which offers a more effective way to secure AI models by treating every interaction as a potential threat.

Prompt injection is a critical vulnerability in many generative AI tools, where attackers craft inputs that manipulate the model’s behavior. This can lead to a range of unintended actions such as data leakage or unauthorized access. For instance, an attacker might embed hidden instructions within a document that, when processed by an LLM, cause it to disclose sensitive information or perform unauthorized operations.

One study by researchers at Bryn Mawr College found that 56 percent of prompt injection tests were successful, illustrating the widespread susceptibility of these models to such attacks. This underscores the urgent need for robust LLM cybersecurity measures to protect AI systems from exploitation.

Zero trust is a modern cybersecurity framework built on a simple principle: never trust, always verify. Instead of assuming that users or systems within the network are safe once they have passed the perimeter, zero trust requires continuous authentication, strict access controls and constant monitoring to validate every interaction.

In practice, this means verifying the identity, intent and context of every request before granting access. Other aspects of this include applying least privilege access by default and assuming that no system, device or user is inherently trusted.

While zero trust is most commonly used to secure networks and user endpoints, its core concepts can also be extended to AI systems. These models face unique risks, including prompt injection attacks, that bypass traditional controls and therefore demand dedicated attention from cybersecurity teams.

By applying zero-trust principles to these models, businesses can limit exposure, control access and ensure each interaction with the system is monitored, filtered and verified. This helps reduce the likelihood of data exfiltration and misuse.

A zero-trust approach assumes that no user, system or input can be trusted by default. The same logic must apply to interactions with generative AI tools. Every prompt entered into an LLM should be treated as potentially hostile and every response should be monitored for signs of manipulation. With AI prompt injection attacks growing in sophistication, businesses must assume these systems will be targeted and design defenses accordingly.

Core zero-trust principles used in network and endpoint security can be adapted for LLM deployments in the following ways:

• Identity verification: Require all users or systems interacting with the LLM to be authenticated using secure credentials or identity providers. This ensures only verified parties can ask questions of the model, as well as keeping an auditable record of user activity.
• Least privilege access: Restrict users to the minimum capabilities required for their role. For example, internal users may have access to business logic, while public users can only submit general prompts. This reduces the risk of accidental or malicious misuse.
• Segmentation and isolation: Treat LLMs as isolated entities within the IT environment. Prevent them from having unrestricted access to internal systems, APIs or sensitive databases unless explicitly necessary and route all interactions through secure gateways.
• Continuous monitoring: Use telemetry and logging tools to observe all activity in real-time. Look for anomalies in prompt structure, output patterns or frequency that may indicate manipulation, misuse or attempted data exfiltration.
• Policy enforcement: Deploy security wrappers or middleware that act as an intermediary between users and the LLM. These tools can enforce input filtering, rate limiting, access control and contextual rules to prevent unsafe or unauthorized interactions.

Applying these principles allows organizations to reduce the risk of prompt injection and treat LLMs as part of a secure, monitored environment.

Prompt hacking attacks can be used to manipulate LLMs into revealing sensitive information that could harm a business. By disguising malicious commands within seemingly harmless prompts, attackers may extract login credentials, financial records or internal model instructions that expose how the system operates. This can be used to gain further access to a network, deliver malware or directly exfiltrate data for use in ransomware.

The risk of exfiltration increases when LLMs are integrated into business systems without strict safeguards. However, applying zero-trust principles helps reduce this threat.
Verifying user identity, restricting access to high-risk functions, filtering prompts and monitoring for unusual activity all limit the opportunity for attackers to exploit the system.

To be effective, LLMs must be treated as part of a broader data protection strategy. Just like servers or user devices, these models are potential targets for cybercriminals. Preventing exfiltration means embedding them into security planning from day one and protecting them with the same scrutiny applied to other critical assets.