Getting started

FAQ

Questions

  1. How's the Privatemode API different from other GenAI APIs?
  2. Can you see my prompts and the corresponding responses?
  3. You run your service on Scaleway. Can't they see my data?
  4. You use the OpenAI API standard. Is any data transferred to OpenAI?
  5. Which data is processed by Edgeless Systems?
  6. Is privatemode.ai down?
  7. Why can't Privatemode reveal data across tenants or users?
  8. Is Privatemode secure against quantum computers and "store now, decrypt later" attacks?

How's the Privatemode API different from other GenAI APIs?

In contrast to other GenAI APIs, the Privatemode API offers dependable privacy and security properties. Your prompts and responses always stay hidden from any third party, even from us, Edgeless Systems.

With other GenAI APIs, data can be accessed by third parties at various points when interacting with the AI. For example:

  • Infrastructure providers like Microsoft or AWS have privileged access and control over the underlying hardware and system software. This means they can potentially access your prompts and responses.
  • GenAI service providers like OpenAI also have the ability to access your data.

With the Privatemode API, your prompts and responses remain confidential. By design, neither Scaleway as the infrastructure provider nor Edgeless Systems as your service provider can access or leak your prompts or responses.

To ensure this, the Privatemode API leverages confidential computing, a cutting-edge technology that provides runtime encryption and protection for data. The Privatemode API applies confidential computing end-to-end and make this verifiable from the outside. With the help of confidential computing-based remote attestation, it's possible to verify the integrity and authenticity of the software of the entire Privatemode software stack.

This makes the Privatemode API fundamentally different from other GenAI APIs. You fully own your prompts and responses.

Feel free to dive deeper into Privatemode's security properties.

Can you see my prompts and the corresponding responses?

No. By leveraging confidential computing and providing hardware-enforced end-to-end encryption, Privatemode ensures that Edgeless Systems can't access your prompts or responses.

You run your service on Scaleway. Can't they see my data?

This is important to us: by design, our infrastructure provider can't access any of your prompts or responses.

The Privatemode API is based on confidential computing, and this is where it truly shines. Thanks to its strong, hardware-enforced confidentiality, even the infrastructure provider that controls the hardware and system software can't access any of your data.

You use the OpenAI API standard. Is any data transferred to OpenAI?

No. The Privatemode API doesn't use any OpenAI services. It only adheres to the common OpenAI interface definitions (prompt and response format) to provide a convenient development experience and ensure easy code portability.

Which data is processed by Edgeless Systems?

Prompts and responses are always encrypted and inaccessible to third parties. For monitoring purposes, the following metadata information is stored for up to 90 days:

  • IP
  • timestamp
  • API key
  • token usage
  • request metadata: path, method, status code
  • response time

Token usage for each API key is permanently stored for billing purposes.

Is privatemode.ai down?

You can check the status of the Privatemode API on the status page.

Why can't Privatemode reveal data across tenants or users?

Privatemode doesn't save prompts or responses after an inference request is completed. The underlying inference engine, vLLM, uses batching to process multiple prompts simultaneously. However, vLLM ensures that requests remain isolated in memory, preventing any cross-tenant interference or data leakage during execution.

Privatemode retains transient inference state in a prompt cache in worker memory. Every cache lookup key includes a secret salt. By default, the Privatemode proxy generates a new random salt for each request, preventing cache reuse. For better performance, clients can opt into cache reuse by using the same salt across requests. Requests with different salts can't reuse each other's cache entries or observe whether a prompt prefix produced a cache hit.

See Prompt cache security for the security design and Secure prompt caching for fast AI inference for a detailed implementation overview.

Is Privatemode secure against quantum computers and "store now, decrypt later" (SNDL) attacks?

Yes. Privatemode is designed to be quantum-resistant and protects against "store now, decrypt later" (SNDL) attacks. We use symmetric memory encryption (AES) for data in-use and quantum-resistant hybrid key exchange (X25519MLKEM768) for data in-transit.