Skip to main content
Version: 1.8

End-to-end prompt encryption

The Privatemode API uses end-to-end encryption to secure user data, ensuring that every component involved in processing an API call only handles encrypted information and can't access the original data. Prompts are encrypted on the client side, decrypted within runtime-encrypted workers, and re-encrypted before being returned to the client.

Secure key exchange​

Privatemode's key exchange protocol has two main goals:

  1. Only initiate key exchange with verified and trusted AI workers.
  2. Establish an end-to-end confidential channel on application level between the client and the AI workers.

Workflow​

  1. Key generation: The client generates a symmetric key and securely stores it in the client's trusted environment.
  2. Secure key upload: The client then interacts with the secret service to securely exchange encryption secrets. This involves the client verifying the Coordinator to ensure that it's interacting with a trusted and verified deployment. Once the Coordinator is verified, the client securely uploads the encryption key to the secret service.
  3. Key distribution to workers: After the Coordinator verifies the AI workers through attestation, the secret service securely distributes the encryption keys to the appropriate AI workers. These workers use the keys to decrypt the prompts, process the data, and then re-encrypt the results before sending them back to the client.

The resulting flow is illustrated below:

Encryption​

The prompt and response encryption uses the exchanged symmetric key with Authenticated Encryption implemented through AES-GCM.

Prompts are encrypted by the client-side privatemode-proxy and decrypted by a server-side encryption proxy hosted on the worker. Responses are handled accordingly where encryption is done by the server-side encryption proxy and decryption of the responses is performed by the client-side privatemode-proxy.

Workflow​

  1. Request Encryption: The client only encrypts the request field with the prompt text, keeping other details like token length accessible to the service provider. The encrypted field encodes the key ID which maps to the used key.
  2. Request Decryption: The server-side proxy decodes the prompt field with the key that maps to the encoded key ID. This doesn't affect the low-level runtime encryption provided by the Confidential Computing Environment.
  3. Prompt Processing: The decrypted prompt is securely transmitted to the inference server locally via a TCP socket.
  4. Response Encryption: The response from the inference server is returned through the same socket. The server-side proxy then encrypts the response and sends it back to the client-side privatemode-proxy.

Service provider isolation​

The Coordinator and AI workers in Privatemode are designed to operate independently of us as the service provider, meaning that we've by no means access to your encryption keys. This security is reinforced during the remote attestation process, where the client not only verifies the Coordinator's integrity but also its identity. By inspecting the open-source code, clients can confirm that the Coordinator is configured to prevent any unauthorized access by the service provider, ensuring that all encryption keys remain secure and exclusively controlled by the client.