RAG or Retrieval-Augmented Generation is a term coined in the 2020 paper: “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” Explained simply, Retrieval Augmented Generation attempts to give more context-specific answers asked of an LLM by augmenting the prompt by retrieving a set of context-specific information.
Security
There are several potential security risks of implementing RAG. Some inherent it RAG, and some by-products of the fact that many retrievers are basically spiders/crawlers.
Retrieval
Data Access Controls
Retrievers are tools to retrieve the data to be added to the prompt, and inform answers given by the LLM. Usually this is accomplished by giving the retriever the desired level of access to the knowledge resource to augment the prompt, and let the retriever follow links and crawl through the data, retrieving it, and converting it to to vector data via an embedding model. Giving the retriever access to a resource not generally available to the public, is likely to expose any places within the data source where data controls are not properly implemented. The retriever will follow all the links it can find, and may surface data, that it was not expected to have access to.
Changes to Data Availability / Data Review
Another issue with this methodology is that data is converted to a vector representation. This means that if some data has been retrieved, and is then made unavailable, it can be tricky to trigger the removal of that data from the vector database. It can also be tricky to evaluate all the data within the vector database, though it is not impossible as we’ll see in the next section.
Vector !== Encryption
Finally, if the vector database is not adequately secured, the vector data can be used to quite accurately recreate a close approximation of the original data. One study found that they were able to recover northwards of 90% of the words in the source material via an embedding. Sources: https://ironcorelabs.com/blog/2023/embeddings-not-human-readable/ https://arxiv.org/pdf/2310.06816.pdf
Hosting
The potential vulnerabilities above can be worsened if the data, that was intended to be private ends up leaking into training data for other models or unintentionally made available in some other way by a vulnerability in a cloud host. The best guard against this seems to be to have the LLMs running within a confidential computing enclave, but getting all aspects of this right can be tricky.
Client data separation
Another consideration in the design of a productized AI system is how to ensure that the data for various clients remains separate and that data does not cross the bounds from client to client.
The Data Itself.
RAG is a process of altering the prompt to include the information that you want to provide the context for your inquiry. This means that the data being retrieved is effectively injected into your prompt. This means that an attacker with the ability to modify the source data, can craft data to influence your prompt. For one example of this, see this report on research into generating an AI worm.