Skip to main content
Version: Next

End-to-end prompt encryption

The Privatemode API uses end-to-end encryption to protect user data from being accessed along the supply chain. Prompts are encrypted on the client side, decrypted within runtime-encrypted workers, and re-encrypted before being returned to the client. This encryption uses a symmetric key with Authenticated Encryption implemented through AES-GCM.

Encryption workflow​

Prompts are encrypted by the client-side proxy and decrypted by server-side encryption proxy hosted on the worker. Replies are handled accordingly where encryption is done by the server-side encryption proxy and decryption of the replies is performed by the client-side proxy. The steps for the encryption between the client and the proxy are:

  1. Key Exchange: The server-side encryption proxy and the client-side proxy exchange a symmetric AES key, facilitated by the attestation service (AS). The AES keys are also referred to as inference secrets in this documentation. The AS handles key uploads from clients and grants access to verified worker nodes. Each key has a corresponding ID which is encoded in the encrypted data to let the decoder know which key was used.
  2. Key Synchronization: The server-side proxy monitors for key updates to stay in sync with the client.
  3. Request Encryption: The client only encrypts the request field with the prompt text, keeping other details like token length accessible to the service provider. The encrypted field encodes the key ID which maps to the used key.
  4. Request Decryption: The server-side proxy decodes the prompt field with the key that maps to the encoded key ID. This doesn't affect the low-level runtime encryption provided by the Confidential Computing Environment.
  5. Prompt Processing: The decrypted prompt is securely transmitted to the inference server via a UNIX domain socket.
  6. Response Encryption: The response from the inference server is returned through the same socket. The server-side proxy then encrypts the response and sends it back to the client-side proxy.