End-to-end prompt encryption

The Privatemode API uses end-to-end encryption to secure user data, ensuring that every component involved in processing an API call only handles encrypted information and can't access the original data. Prompts are encrypted on the client side, decrypted within runtime-encrypted workers, and re-encrypted before being returned to the client.

Secure key exchange

Privatemode's key exchange protocol has two main goals:

1. Only establish keys with a verified deployment, and release them only to attested AI workers.
2. Establish an end-to-end confidential channel on application level between the client and the AI workers.

Workflow

1. Coordinator verification: The client connects to the Privatemode backend to verify the identity and integrity of the Contrast Coordinator through remote attestation. This ensures the client is talking to a genuine, untampered Privatemode deployment.
2. Root certificate retrieval: Upon successful verification, the backend provides the Coordinator Mesh CA certificate to the client. This certificate acts as the trust anchor for the entire deployment.
3. Key agreement: The client agrees on a symmetric encryption key with the secret service using HPKE with the hybrid post-quantum KEM X25519MLKEM768:
   • The client generates an ephemeral key pair and sends the public key to the secret service through the Privatemode HTTP API.
   • The secret service derives a shared secret for the client's public key and returns the encapsulated key together with its certificate, signing both with its private key. The certificate is issued by the Mesh CA.
   • The client verifies the certificate against the Mesh CA, checks the signature, and derives the same shared secret from the encapsulated key.
4. Key distribution to workers: After the Coordinator verifies the AI workers through attestation, the secret service securely distributes the encryption keys to the appropriate AI workers. These workers use the keys to decrypt the prompts, process the data, and then re-encrypt the results before sending them back to the client.

Each key is valid for one hour. The secret service discards it afterward, and the client establishes a new key through the same exchange. The secret service and the AI workers keep keys inside their attested confidential VMs and never write them outside that environment.

The resulting flow is illustrated below:

flowchart LR
  A[Client software]-- "verifies" -->B[Coordinator]
  B[Coordinator]-- "verifies" -->C[secret service]
  B[Coordinator]-- "verifies" -->D[AI workers]
  A[Client software]-- "agrees on keys" -->C[secret service]
  C[secret service]-- "sends keys" -->D[AI workers]

Encryption

The prompt and response encryption uses the exchanged symmetric key with Authenticated Encryption implemented through AES-GCM.

Prompts are encrypted by the client-side Privatemode proxy and decrypted by a server-side encryption proxy hosted on the worker. Responses are handled accordingly where encryption is done by the server-side encryption proxy and decryption of the responses is performed by the client-side Privatemode proxy.

Workflow

1. Request encryption: The client encrypts the request body, which includes the prompt and other values such as the model parameters. HTTP headers stay in plaintext. They carry the metadata required for routing, billing, and authentication:

   • Privatemode-Target-Model: the name of the requested model.
   • Privatemode-Content-Length-Tokens: the client's estimate of the request length in tokens.
   • Privatemode-Secret-ID: the ID of the key that the client used, which the server-side proxy maps to the key.
   • Authentication, versioning, and encryption handshake headers, for example Authorization and Privatemode-OAE-Request-Header.

   Plaintext metadata lists what each endpoint sends in these headers.

2. Request decryption: The server-side proxy decrypts the body with the key that maps to the key ID from the header. This doesn't affect the low-level runtime encryption provided by the Confidential Computing Environment.

3. Prompt processing: The decrypted request is securely transmitted to the inference server locally via a TCP socket.

4. Response encryption: The response from the inference server is returned through the same socket. The server-side proxy then encrypts the response body and sends it back to the client-side Privatemode proxy. A non-streaming response is sealed as a single encrypted body. A streaming response (server-sent events) is sealed event by event, so the client can decrypt and process each event as it arrives.

Plaintext metadata

The service can't read request or response bodies, but it needs some metadata to route a request to a worker that serves the requested model, to apply rate limits, and to bill usage. The client-side proxy derives this metadata from the request before encrypting it and sends it in the headers listed above. The following table lists what the service sees per endpoint.

Shard key: With prompt caching enabled, or when a request carries its own cache_salt, the client also sends the Privatemode-Shard-Key header in plaintext. The API gateway uses it to route requests that share a prompt prefix to the worker holding the matching cache entries. The shard key starts with a hash of the cache salt, followed by one character per block of the prompt. Each character is taken from a SHA-256 hash chain that starts with the salt and covers the prompt up to that block, so the header reveals neither the salt nor the prompt. The service can tell that two requests within the same cache scope share a prefix, and roughly how long that prefix is.

Service provider isolation

The Coordinator, the secret service, and the AI workers run in attested confidential environments that operate independently of Edgeless Systems as the service provider, so Edgeless Systems has no access to your encryption keys. The remote attestation step reinforces this: the client verifies not only the Coordinator's integrity but also its identity. By inspecting the public source code, clients can confirm that the deployment is configured to prevent any access by the service provider, so that encryption keys exist only on the client and inside attested confidential environments.