Security

Security

This page provides an overview of Privatemode's security properties. If user privacy and data protection are non-negotiable for you, the Privatemode API is the right choice.

Security properties

The Privatemode API delivers robust security features:

  • Confidentiality: By design, all your prompts and responses remain private, meaning they're accessible only to you. Privatemode leverages confidential computing to enforce end-to-end encryption from the application, through the inference process by the GenAI, and back to the user.

  • Integrity and authenticity of the GenAI service: Strong isolation within hardware-enforced Confidential Computing Environments (CCEs), combined with integrity and authenticity verification of the entire API infrastructure and code, ensures a thorough protection against tampering or malicious manipulation.

You can find details on how these security goals are achieved in the Architecture section. For jurisdiction, contracts, certifications, and data retention, see Trust and compliance.

Verifiability

Privatemode's security and services are designed to be transparently verifiable. This is ensured by:

  1. Public source code combined with reproducible builds
  2. Provider-independent, hardware-enforced cryptographic hashes of the service–enabled by CCEs

(1) allows API users to audit the service and generate reference values, while (2) enables verification against the actual provided service.

This approach is fundamentally different from other GenAI services, as it provides truly verifiable confidentiality. See the Verification from source code and Verification of model integrity guides for more details.

Threat model

This section describes what Privatemode protects, against whom, and under which assumptions. It also states what's out of scope, so you can assess the residual risk for your use case.

Assets

Privatemode protects the following assets:

  • Prompts and responses: everything you send to and receive from a model, including uploaded files, images, and audio, as well as the contents of tool calls and structured outputs.
  • Inference state: intermediate data derived from your prompts during processing, such as the prompt cache.
  • Encryption keys: the keys that protect prompts and responses between your client and the AI worker.
  • The served software and model: the integrity of the code and model weights that process your prompts. A tampered inference stack or model could leak or manipulate data even inside an isolated environment.

Request metadata, such as the model name, token counts, timestamps, and your API key, isn't protected in this sense. The service needs it for routing, rate limiting, and billing. See Out of scope.

Parties in a GenAI API call

Sketch of entities

To help you understand who can typically access user data in conventional GenAI API services, we provide an overview of the usual parties involved in the supply chain and explain why they often have access to your data.

In most GenAI API services, the following four relevant entities are involved and have direct or indirect access to certain types of sensitive data:

  • The infrastructure provider: Provides the compute infrastructure to run the model and inference code, such as AWS or CoreWeave.
  • The platform provider: Supplies the software environment that runs the AI model, such as Hugging Face.
  • The model provider: Develops and/or supplies the actual AI model, such as Mistral or Anthropic.
  • The service provider: Integrates all components and offers the SaaS to the user.

In many scenarios, one organization may have different roles at the same time. The following table gives three examples.

APIService providerPlatform providerModel providerInfrastructure provider
OpenAI GPTOpenAIOpenAIOpenAIMicrosoft Azure
HuggingFaceHuggingFaceHuggingFaceCohere, Mistral, and othersAWS, GCP, and others
PrivatemodeEdgeless SystemsvLLMOpen-weights model providersScaleway, Lyceum

In the case of the well-known OpenAI GPT API, OpenAI is the service provider, the platform provider, and the model provider, while Microsoft Azure provides the infrastructure.

HuggingFace offers an inference API, which allows the user to choose between AI models. The company HuggingFace acts both as the service provider and the platform provider.

The Privatemode API is provided by us (Edgeless Systems). The service runs on infrastructure from Scaleway and Lyceum Technology in the EU and uses the open-source framework vLLM to serve open-weights AI models from various providers.

What conventional GenAI APIs expose

Let's examine how these entities can access relevant data within widespread GenAI services like the OpenAI API and the HuggingFace API.

The infrastructure provider is highly privileged and controls hardware components and system software like the hypervisor. With this control, the infrastructure provider can typically access all data that's being processed. In the case of a GenAI API service, this includes the user data and the AI model.

On top of the infrastructure runs the software provided by the platform provider. This software has access to both the AI model and the user data. The software may leak data through implementation mistakes, logging interfaces, remote-access capabilities, or even backdoors.

The service provider typically has privileged access to the platform software and the software (e.g., a web frontend) that receives user data. Correspondingly, the service provider can access both the AI model and the user data. In particular, the service provider may decide to re-train or fine-tune the AI model using the user data. This is oftentimes a concern among users, as it may leak data to other users through the AI model's answer. For example, such a case has been reported for ChatGPT.

In the simplest case, the model provider only provides the raw weights (i.e., numbers) that make up the AI model. In this case, the model provider can't, directly or indirectly, access user data. However, in cases where the model provider provides additional software, leaks similar to those discussed for the platform provider may happen for user data.

What Privatemode protects against

Privatemode removes each party's access to your data with a specific, verifiable mechanism.

PartyAccess in a conventional GenAI APIHow Privatemode prevents it
Infrastructure providerControls the hypervisor, memory, and hardware, so it can read all data in plaintext.The Coordinator, the secret service, and the AI workers run in confidential VMs with hardware-enforced memory encryption and isolation (AMD SEV-SNP or Intel TDX, plus NVIDIA confidential computing on the GPU). Your client verifies this through remote attestation before it sends any data, and it accepts evidence only from the machines listed in the manifest.
Platform provider (vLLM)Its inference software handles prompts in plaintext and may leak them through logging, remote access, or backdoors.The platform provider operates nothing in Privatemode. Its software runs inside attested workers as a container image pinned by digest and runtime policy, so it can't be swapped or extended, and its sources are public.
Model providerSupplies weights, and sometimes additional software with access to prompts.Privatemode uses only the model weights. They're mounted read-only with dm-verity integrity checks that are anchored in attestation. No model-provider software runs in Privatemode.
Service provider (Edgeless Systems)Has operational access to the platform software and the frontend, so it can read prompts, log them, or train on them.Prompts and responses are end-to-end encrypted between your client and the attested worker. Keys are established with the attested secret service and released only to attested workers. Edgeless Systems operates the deployment through the untrusted Kubernetes layer and has no access to CVM memory or keys. Any change to the deployment changes the manifest and is visible to your client.
Other tenantsShare the inference service and could probe shared state such as a prompt cache.Requests are isolated in worker memory, and the prompt cache is salted per client, so entries can't be reused or probed across tenants. See Prompt cache security.
Network attackerCan intercept traffic between you and the service.Requests and responses are encrypted with AES-GCM under keys from a hybrid post-quantum key exchange, inside TLS.
Attacker who redirects traffic to another machineCan route your requests to a confidential-computing machine under their control that passes a generic attestation check.The manifest pins the physical machines Privatemode runs on through their hardware identifiers, so evidence from any other machine is rejected even if it's otherwise valid. See Allowed machines.

Trust assumptions

Privatemode's guarantees rest on the following assumptions. Each item states how you can verify it.

  1. Hardware roots of trust. The confidential-computing features of AMD, Intel, and NVIDIA hardware and their public key infrastructures work as specified. Attestation reports chain to the vendors' roots, the manifest enforces minimum firmware and TCB versions, and published vulnerabilities are tracked on the Hardware integrity status page.
  2. Your client component is genuine. The Privatemode proxy or the SDK is the trust anchor: it performs the verification and holds your encryption keys. Obtain it from the official sources, pin the proxy's container image by digest, or build it from source.
  3. Reference values. By default, your client fetches the manifest published by Edgeless Systems. In this mode, Edgeless Systems can roll out a new version, and your client accepts it once the deployment matches the new manifest. The change isn't blocked. The proxy records every accepted manifest in its local audit log. If your policy requires that no change is accepted without your review, reproduce the manifest from source and pin it. Your client then refuses any deployment that differs, and Edgeless Systems is removed from the trust chain. See Change visibility.
  4. Correctness of the trusted code. Contrast, vLLM, and Privatemode's own components run inside the attested environment. A vulnerability in them could affect confidentiality. The code is public and builds reproducibly, so it can be audited.
  5. Web app. The hosted web app extends the trusted computing base to Edgeless Systems and Cloudflare, which deliver the client code to your browser. Self-host the app to avoid this.
  6. Your environment. The machine that runs the client component, your API keys, and the network between your application and the proxy are under your control. Run the proxy on the same machine as your application or in a trusted network with TLS.

Out of scope

The following isn't covered by Privatemode's guarantees:

  • Metadata. The plaintext metadata sent with each request, such as the model name and an estimated token count, and request metadata such as IP address, timestamp, API key, and response time are visible to the service. Trust and compliance lists what's retained and for how long. The sizes and timing of encrypted messages can also reveal approximate prompt and response lengths.
  • Availability. The infrastructure and service providers can interrupt the service. The guarantees concern confidentiality and integrity, not availability. See the status page.
  • Compromise on your side. Malware on the machine that runs the client component, or a tampered client binary, can read your data before it's encrypted. A leaked API key lets someone send requests billed to your organization, but not read your prompts, because the encryption keys are held by your client.
  • Model behavior. The quality of model outputs, hallucinations, and prompt injection through content you feed to the model are outside the threat model. Tool calls are executed by your application, not by Privatemode.
  • Physical access to the hardware. An attacker with physical access to the machines, for example to interpose on memory buses, is outside the threat model. Published hardware attacks of this kind and their mitigation status are tracked on the Hardware integrity status page.
  • Unmitigated hardware vulnerabilities. Newly discovered attacks on confidential-computing hardware are a residual risk until they're mitigated. The Hardware integrity status page states the status per vulnerability.