Overview
The Privatemode API is a ChatGPT-like GenAI API that enables secure processing of prompts using end-to-end encryption and confidential-computing environments. For you as an API user, this means that your prompts to the AI remain private. The same applies to the AI's responses. "Private" in this context means inaccessible to anyone but you.
This overview is meant to give you a deeper technical understanding of how the Privatemode API achieves its unique security properties.
Privatemode's approach
The Privatemode API ensures that your data remains inaccessible to anyone but you. This means that from the moment you make an API call at the client, through prompt processing by the AI model, and back to the client, your data stays protected. Additionally, Privatemode ensures that none of the involved services can send your data elsewhere.
Privatemode achieves this by ...
- on the server side, creating a seamless and comprehensive confidential-computing environment (CCE), which includes all services that access your data during the processing of an API call. The CCE prevents access from the outside and leaks from the inside.
- on the client side, verifying the integrity and authenticity of the server-side CCE using a mechanism called "remote attestation." Based on this, the client side and the server-side CCE exchange cryptographic keys, which the client side uses to encrypt your prompt and the server-side CCE uses to encrypt the corresponding response.
If confidential computing and concepts like remote attestation are new to you, check out our Confidential Computing Wiki to learn more. In a nutshell, confidential computing is a technology that keeps data encrypted in memory—even during processing—and allows you to verify the integrity of workloads. This is ultimately enforced through special hardware extensions in modern processors and accelerators.
Abstracting confidential computing
The Privatemode API is designed for seamless usage and easy integration, handling all the complexities of confidential computing behind the scenes. On the client side, the key component is the privatemode-proxy, which manages remote attestation and end-to-end encryption.
Since this proxy deals with any communication to the server side, we refer to it as the client software or simply the client. Ideally, it runs on the user's machine to ensure the trust anchor excludes any additional parties.
Verifying confidentiality
Remote attestation is a core feature of the Privatemode API, ensuring the confidentiality and integrity of the GenAI endpoint.
Remote attestation: Remote attestation is a cornerstone of confidential computing, and it plays a critical role in Privatemode. In the context of Privatemode, the client uses remote attestation to verify that all server-side software components are both trustworthy and in their intended state. By leveraging independent cryptographic certificates and hardware-enforced signatures, remote attestation ensures that the GenAI endpoint is genuinely confidential, securely isolated, and running valid, trusted AI code. Successful remote attestation is always the necessary precondition for any key exchange and prompt transfer.
Attestation service: To ensure scalability, the client itself only verifies Privatemode's attestation service. This service, running in a CCE, handles the verification of the actual AI workers within the GenAI endpoint.
To learn more about attestation in Privatemode, visit the dedicated section.
End-to-end encryption
By verifying the server side through remote attestation, the client ensures that prompt encryption keys are securely derived and stored. These keys are never shared with anyone except the privatemode-proxy on the client side and the AI worker running completely isolated within a CCE on the server side.
Using modern encryption schemes to secure all prompts and replies, this ensures a confidential channel between the user and the AI.
You can find more details in our encryption section.
Protection against the infrastructure
The Privatemode API uses confidential computing to shield the AI worker that processes your prompts on the server side. Essentially, the AI worker is a virtual machine (VM) that has access to an AI accelerator like the Nvidia H100 and runs some AI code. The AI code loads an AI model onto the accelerator, pre-processes prompts, and feeds them to the AI model. Privatemode applies confidential computing to both VM and the AI accelerator and establishes a secure connection between the two.
With this approach, Privatemode shields the AI worker (and all data it processes) from the rest of the infrastructure. Here, "the infrastructure" includes the entire hardware and software stack that the AI worker runs on, as well as the people managing that stack.
Currently, in many cases, Privatemode's server-side components run on Microsoft Azure. Thus, Microsoft Azure is "the infrastructure" and Privatemode's use of confidential computing ensures that Microsoft Azure can't access any of your data.
Protection against Edgeless Systems
We, at Edgeless Systems, are your GenAI SaaS provider. Confidential computing ensures that GenAI endpoints operate in a fully isolated environment. Independent cryptographic certificates and key material are used to establish the CEE. This setup is verifiable by the client through remote attestation and guarantees that the endpoints are trustworthy and can't be manipulated by us.
The Privatemode API further ensures that all data exchanged with the AI is end-to-end encrypted. Prompts and replies remain completely private.
Our open-source approach works hand-in-hand with confidential computing. Together, they establish a verifiable and confidential channel between you and the GenAI endpoint.
This design ensures that we can never access your data.