The point where LLM attacks cause material damage is the lethal trifecta: an agent holding access to private data, exposure to untrusted content, and a channel to act externally can be directed by injected instructions to exfiltrate private data, and removing any one capability reduces exposure.

The point at which LLM attacks cause material damage has a specific structure and is identifiable in a system. It is also called the lethal trifecta. It consists of three capabilities held together by a single agent: Access to private data ... Exposure to untrusted content ... A channel to send data out or act externally ... An agent holding all three can be directed by injected instructions to transfer private data to an attacker. Model alignment does not remove this exposure, because producing output that conforms to instruction-like input is how the model normally operates. Removing any one of the three capabilities reduces the exposure. The least costly reduction is usually cutting the outbound channel.

Read full article ↗excerpt · fair-use quotation