Privatemode proxy

The Privatemode proxy exposes a local HTTP endpoint that connects applications in any programming language and tools such as coding agents to the Privatemode API. It:

1. Verifies the Privatemode deployment at api.privatemode.ai, where encrypted prompts are processed by the GenAI. The verification process is described in the attestation section.
2. Transparently encrypts user prompts and decrypts responses from the Privatemode API.

The Privatemode proxy is published as a Docker image on GitHub.

Common setups

The following setups cover most deployments.

Local use

Run the proxy on your machine with your API key and point your application at http://localhost:8080/v1:

docker run -p 127.0.0.1:8080:8080 ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest --disable-prompt-cache=false --api-key-fallback=always --api-key=<your-api-key>

The proxy verifies the deployment at startup and holds the key, so the API key field of your application only needs a placeholder value. See API key.

Shared proxy for a team or a server

Run the proxy on a host in a trusted network and let several applications or users share it. Pin the image by digest, read the API key from a mounted file, serve TLS to your clients, share the prompt cache across users, and keep the manifest log on a volume:

docker run -p 8080:8080 \
  -v "$PWD/api-key.txt:/run/secrets/privatemode-api-key:ro" \
  -v "$PWD/certs:/certs:ro" \
  -v "$PWD/proxy-workspace:/app/privatemode-proxy" \
  ghcr.io/edgelesssys/privatemode/privatemode-proxy@sha256:<digest> \
  --api-key=@/run/secrets/privatemode-api-key \
  --tls-cert-path=/certs/server.crt --tls-key-path=/certs/server.key \
  --disable-prompt-cache=false \
  --workspace=/app/privatemode-proxy

See Running the container for pinning, Setting up TLS, Prompt caching for the salt options, and Manifest management for the log on the workspace volume.

Per-request API keys

Start the proxy with --api-key-fallback=never. Each client then sends its own key in the Authorization header, which the proxy forwards to the Privatemode API. Use this when departments or cost centers need separate keys behind one proxy. See Using multiple API keys.

Kubernetes

Deploy the proxy with the Helm chart. It covers the API key secret, TLS, the shared prompt cache, a persistent workspace, and custom CA bundles.

Running the container

The following command starts the Privatemode proxy and exposes it on localhost port 8080:

docker run -p 127.0.0.1:8080:8080 ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest

Omit 127.0.0.1: if you want to expose the port on all network interfaces.

API key

The Privatemode API requires authentication with the API key you received when you signed up. Providing it to the proxy with the --api-key flag lets the proxy handle authentication. The flag accepts the key itself or a file path prefixed with @, for example --api-key=@/run/secrets/privatemode-api-key, which keeps the key out of the process list and shell history.

Use --api-key-fallback to control when the configured key replaces request authentication:

• never: Forward request authentication unchanged. The configured key is still used for attestation.
• always (default): Use the configured key for every forwarded request, replacing any client-provided API key.
• if-placeholder: Use the configured key only when the API key in the request is placeholder (the Authorization header is exactly Bearer placeholder). Missing headers and other API keys are forwarded unchanged.

Without a configured --api-key, all modes forward request authentication unchanged.

Using multiple API keys

If you need to support multiple API keys (e.g., for different departments or cost centers), you have two options:

1. Pass the API key in the request: Start the proxy without the --api-key flag, or use --api-key-fallback=never to reserve the configured key for attestation. The proxy will forward the Authorization header from your client's request to the Privatemode API. This allows each client to provide their own API key.

   from openai import OpenAI
   import os
   
   client = OpenAI(
       # Pass the API key here
       # A placeholder suffices when the proxy runs with --api-key.
       api_key=os.environ.get("PRIVATEMODE_API_KEY", "placeholder"),
       # Point to your local proxy instance
       base_url="http://localhost:8080/v1",
   )
   
   completion = client.chat.completions.create(
       model="gpt-oss-120b", messages=[{"role": "user", "content": "Hello Privatemode!"}]
   )

2. Run multiple proxy instances: Deploy a separate proxy instance for each API key, configuring each with a different --api-key and listening on a different port.

CLI flags

To see all available CLI option flags, use:

docker run ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest --help

Options

      --api-endpoint string                         The endpoint for the Privatemode API (default "api.privatemode.ai:443")
      --api-key string                              The API key for the Privatemode API. Accepts either a direct literal or a file path prefixed with '@'. If no key is set, the proxy will not authenticate with the API.
      --api-key-fallback string                     When to use the configured API key for forwarded requests: never, always, or if-placeholder (only when the API key in the request is 'placeholder'). The default will change to never in a future release. (default "always")
      --cors-allowed-origins strings                Allowed browser origins (comma-separated). Empty disables CORS; '*' allows any origin.
      --disable-prompt-cache                        Use a new random salt per request instead of sharing a cache across the proxy. Request cache_salt can still be used to achieve shared caches. (default true)
      --dump-requests                               If set, the proxy dumps all requests and responses in plaintext to the '/requests' sub‑directory of the workspace. Only use this for debugging on a machine you control.
  -h, --help                                        help for privatemode-proxy
      --listen-address string                       The IP address or hostname on which the proxy listens for incoming API requests. (default "0.0.0.0")
      --log-format string                           set logging format (json or text) (default "text")
  -l, --log-level string                            set logging level (debug, info, warn, error, or a number) (default "info")
      --manifest-path string                        The path for the manifest file. If not provided, the manifest will be read from the remote source.
      --nvidia-ocsp-allow-unknown                   Whether it should be tolerated if the NVIDIA OCSP service cannot be reached. (default true)
      --nvidia-ocsp-revoked-grace-period int        The grace period (in hours) for which to accept NVIDIA attestation certificates that are revoked according to the OCSP service. Supplying a value of 0 disables the grace period, meaning that revoked certificates are rejected immediately. (default 48)
      --port string                                 The port on which the proxy listens for incoming API requests. (default "8080")
      --prompt-cache-salt string                    The salt used to separate prompt caches. Can either be a literal, @file, or random (one salt per proxy startup). Requires --disable-prompt-cache=false. Request cache_salt takes precedence. Literal and file salts must be valid UTF-8, at least 32 bytes and at most 128 characters, without '@', '/', '\', or NUL. (default "random")
      --timeout-idle duration                       Maximum time to wait for the next request on a keep-alive connection. Zero uses the value from --timeout-read. (default 1m0s)
      --timeout-read duration                       Maximum time to read an entire incoming request, including its body. Zero disables the timeout. (default 2m0s)
      --timeout-read-header duration                Maximum time to read incoming request headers. Zero uses the value from --timeout-read. (default 10s)
      --timeout-upstream-response-header duration   Maximum time to wait for upstream response headers after sending the request. Zero disables the timeout. (default 1h0m0s)
      --timeout-write duration                      Maximum time to write a response. Zero disables the timeout.
      --tls-cert-path string                        The path to the TLS certificate. If not provided, the server will start without TLS.
      --tls-key-path string                         The path to the TLS key. If not provided, the server will start without TLS.
      --workspace string                            The path into which the binary writes files. This includes the manifest log data in the 'manifests' subdirectory. (default ".")

Logging

The proxy logs to standard error. Use --log-level (debug, info, warn, error) for the level of detail and --log-format=json for structured logs that log collectors can parse.

Setting up TLS

If the Privatemode proxy runs on another machine and is accessed over a network, configure TLS to encrypt the traffic between the application and the proxy.

To enable TLS, provide a certificate and a private key to the container. Mount the directory containing these files into the container so the proxy can access them.

Use the following flags to specify the paths inside the container:

• --tls-cert-path: Path to the TLS certificate file.
• --tls-key-path: Path to the private key file.

For example, if your certificates are in the current directory on your host, you can mount it to /certs inside the container:

docker run \
  -p 8080:8080 \
  -v "$PWD:/certs" \
  ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest \
  --tls-cert-path=/certs/server.crt \
  --tls-key-path=/certs/server.key

By providing these, the Privatemode proxy will serve traffic from and to your application client via HTTPS, ensuring secure communication. If these flags aren't set, the Privatemode proxy will fall back to serving traffic over HTTP.

CORS

CORS is disabled by default. Enable browser access with --cors-allowed-origins=https://client.example, a comma-separated list of exact origins, or --cors-allowed-origins='*' for any origin. The proxy handles preflight requests automatically and allows browsers to cache successful preflight responses for up to 24 hours.

Outbound network traffic

When running the Privatemode proxy in an environment with restricted firewall settings, you might need to allow the following domains and ports:

• cdn.confidential.cloud:443: for fetching the latest manifest
• api.privatemode.ai:443: for communication with the Privatemode backend

HTTP proxy

You can run the Privatemode proxy behind an HTTP proxy that supports HTTP CONNECT. Set the https_proxy environment variable like this:

docker run -p 127.0.0.1:8080:8080 -e https_proxy=<proxy-address> ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest

TLS inspection / MitM appliances

If your network uses a TLS inspection or Man-in-the-Middle (MitM) appliance, the Privatemode proxy might fail to verify the appliance-issued certificates for outbound connections to cdn.confidential.cloud or api.privatemode.ai.

In this case, mount a CA bundle into the container and point SSL_CERT_FILE to it.

Docker

Create or obtain a PEM bundle, then mount it into the container:

docker run \
  -p 8080:8080 \
  -v /path/to/ca-bundle.crt:/etc/ssl/certs/custom-ca-bundle.crt:ro \
  -e SSL_CERT_FILE=/etc/ssl/certs/custom-ca-bundle.crt \
  ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest

To verify that the configuration works, restart the container and check its logs. Startup should include Coordinator manifest fetched successfully, and errors like x509: certificate signed by unknown authority should no longer appear.

Prompt caching

The proxy can share a prompt cache across the clients it serves. See Prompt caching for how caching works, per-request salts, and how to choose and handle salts, and Prompt cache security for the isolation design.

Modes of operation

By default, the proxy generates a new random salt for each request without its own cache_salt, so requests don't share a cache.

Set --disable-prompt-cache=false to share a cache across all clients of the proxy instance. --prompt-cache-salt sets the shared salt:

• random (default): one random salt for the lifetime of the proxy, so the cache is shared until restart.
• A literal salt: a fixed salt, shared across proxy instances and restarts.
• @file: read a fixed salt from a file, trimming surrounding whitespace. For example, generate the file with openssl rand -hex 32 > cache-salt and pass --prompt-cache-salt=@cache-salt.

Literal and file salts must be valid UTF-8, contain at least 32 bytes and at most 128 Unicode characters, and exclude @, /, \, and NUL.

A cache_salt in the request always takes precedence, including when --disable-prompt-cache is set.

HTTP timeouts

You can customize HTTP timeouts if required.

Zero turns off the read, write, and upstream response-header timeouts. For read-header and idle timeouts, zero falls back to --timeout-read. Set that to zero as well to turn them off.

The write timeout is off by default so long inference responses can complete.

Manifest management

The proxy verifies the Privatemode deployment against a manifest, which pins the code, configuration, and hardware the deployment may run. The manifest page explains what it contains and how a pinned manifest differs from the default mode. This section covers how to configure the proxy for each mode.

By default, the manifest is managed automatically. Manual control requires extra work each time an update to the Privatemode API is rolled out.

Automatically

By default, the Privatemode proxy fetches a manifest from a file service managed by Edgeless Systems (you can get it here). Whenever validation of the Privatemode deployment fails, the Privatemode proxy fetches the latest manifest from the file service and retries validation. This allows the proxy to continue working without manual intervention, even if the deployment changes.

To ensure auditability of the enforced manifests over time, changes to the manifest are logged to the local file system. These logs serve as a transparency log, recording which manifest was used at what point in time to verify the Privatemode deployment.

The proxy writes a file called log.txt. For each manifest that's enforced by the proxy, log.txt contains a new line with the timestamp at which enforcement began and the filename of the manifest that was enforced.

log.txt and the corresponding manifests are stored in a folder manifests. You can use the CLI flag --workspace to control where the folder manifests is stored.

Mounting the workspace to the Docker host preserves this transparency log when the container is removed:

docker run -p 127.0.0.1:8080:8080 -v proxy-logs:/app/privatemode-proxy ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest --workspace /app/privatemode-proxy

Manually

You can generate a manifest manually and provide the file path to the Privatemode proxy via its --manifest-path CLI flag.

NVIDIA OCSP

You can set the policy for handling responses of the NVIDIA OCSP using the following flags:

• nvidia-ocsp-allow-unknown: Whether the "unknown" OCSP status (i.e., OCSP is unreachable or doesn't provide information about this certificate) should be tolerated. (Default: true)
• nvidia-ocsp-revoked-grace-period: How long "revoked" OCSP responses should be accepted for after the revocation time, in hours. A value of 0 means that "revoked" OCSP responses aren't accepted at all. (Default: 48)

For a more detailed explanation of the policy, see the documentation on certificate revocation.

Extract a static binary

If you want to run the proxy as a binary, you can extract it from the container image. Depending on your architecture (arm64 or amd64), insert the <arch> variable below to obtain a static Linux binary like this:

containerID=$(docker create --platform=linux/<arch> ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest)
docker cp -L "${containerID}":/bin/privatemode-proxy ./privatemode-proxy
docker rm "${containerID}"

Helm chart

You can use the privatemode-proxy Helm chart for easy deployment to Kubernetes.

Prerequisites

• Kubernetes 1.16+
• Helm 3+
• (Optional) Persistent Volume for workspace
• (Optional) ConfigMap for manifest file
• (Optional) TLS secret for certificates

Installation

You can install the chart with the following commands:

helm repo add edgeless https://helm.edgeless.systems/stable
helm repo update

helm install privatemode-proxy edgeless/privatemode-proxy

Configuration

API key

Store the API key in a Kubernetes secret. For example:

kubectl create secret generic privatemode-api-key --from-literal=apiKey=your-api-key

Set config.apiKey.fallback to never, always, or if-placeholder to control when the configured key replaces request authentication. The default empty value omits the flag and uses the proxy's default. See API key for the modes' behavior.

For example, to use the configured key only when the API key in the request is placeholder:

config:
  apiKey:
    fallback: 'if-placeholder'

CORS

Set config.corsAllowedOrigins to a list of exact browser origins, or ['*'] to allow any origin. The default empty list disables CORS.

config:
  corsAllowedOrigins:
    - 'https://client.example'

See CORS for preflight behavior.

Shared prompt cache

Cache sharing is disabled by default. Set config.sharedPromptCache.enabled=true to share a cache across clients of the proxy instance. Client-provided salts always take precedence.

To share prompt caches across proxy instances, store a random salt of at least 32 bytes in a Kubernetes Secret:

openssl rand -hex 32 | kubectl create secret generic privatemode-proxy-prompt-cache-salt --from-file=promptCacheSalt=/dev/stdin

Then, configure these values for your chart:

config:
  sharedPromptCache:
    enabled: true
    promptCacheSalt:
      enabled: true
      secretName: 'privatemode-proxy-prompt-cache-salt'

Persistent volume

To persist the application’s data beyond the lifetime of the current deployment, you can configure a Persistent Volume. The data includes the transparency log and manifests that allow you to audit the enforced manifests over time.

First, create a PersistentVolumeClaim:

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: privatemode-proxy-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Then, configure these values for your chart:

config:
  workspace:
    enabled: true
    volumeClaimName: "privatemode-proxy-pvc"

TLS configuration

To enable TLS for communication between your application and the Privatemode proxy, provide the TLS certificate and key through a Kubernetes secret:

You can use cert-manager to manage the TLS secret. Or you can create it manually as follows:

kubectl create secret tls privatemode-proxy-tls \
  --cert=<path-to-cert> --key=<path-to-key>

Then, configure these values for your chart:

config:
  tls:
    enabled: true
    secretName: "privatemode-proxy-tls"

TLS inspection / MitM appliance CA bundle

If your cluster reaches Privatemode through a TLS inspection appliance, create a secret that contains a PEM CA bundle:

kubectl create secret generic privatemode-proxy-custom-ca \
  --from-file=ca-bundle.crt=/path/to/ca-bundle.crt

Then, configure these values for your chart:

config:
  customCA:
    enabled: true
    secretName: "privatemode-proxy-custom-ca"
    fileName: "ca-bundle.crt"

If the bundle is already available as a ConfigMap, for example one managed by cert-manager's trust-manager, set configMapName instead. It takes precedence over secretName. Set fileName to the ConfigMap key that holds the bundle:

config:
  customCA:
    enabled: true
    configMapName: "corporate-ca-bundle"
    fileName: "ca-bundle.crt"

When enabled, the chart mounts the bundle and sets SSL_CERT_FILE for the container. The bundle is read at startup, so restart the proxy after updating it:

kubectl rollout restart deployment/privatemode-proxy-<release-name>

HTTP proxy configuration

You can run the Privatemode proxy behind an HTTP proxy that supports HTTP CONNECT. To this end, set the https_proxy environment variable for your chart:

config:
  extraEnv:
    - name: https_proxy
      value: <proxy-address>

HTTP timeouts

Pass timeout flags through config.extraArgs in your Helm values file:

config:
  extraArgs:
    - '--timeout-read=5m'
    - '--timeout-upstream-response-header=30m'

Manifest file configuration

While manually managing manifests isn't recommended (see Manifest management), you can pass in the manifest via a ConfigMap:

Create the ConfigMap from your manifest file:

kubectl create configmap privatemode-proxy-config --from-file=manifest.json=/path/to/your/manifest.json

Then, configure these values for your chart:

config:
  manifest:
    enabled: true
    configMapName: "privatemode-proxy-config"
    fileName: "manifest.json"
    mountPath: "/etc/config/manifest.json"

Accessing the proxy

Once the deployment is complete, you can configure your application to access the API through the Privatemode proxy service’s domain.

By default, the proxy can be accessed at the following URL:

http://privatemode-proxy-privatemode-proxy.default.svc.cluster.local:8080/v1

This URL is constructed as follows:

http://privatemode-proxy-{helm-release}.{namespace}.svc.cluster.local:{port}/v1

• {helm-release}: The name of your Helm release.
• {namespace}: The Kubernetes namespace where the proxy is deployed.
• {port}: The port configured for the proxy service (default is 8080).

If you configured a custom DNS entry in your cluster, adjust the URL accordingly. Replace the default service domain with your custom domain, ensuring that your application can correctly resolve and communicate with the Privatemode proxy service.

Uninstallation

You can uninstall the chart as follows:

helm uninstall privatemode-proxy

Troubleshooting

Requests fail with HTTP status 500

The proxy returns status 500 with a message starting with forwarding request: getting latest secret: when it holds no valid encryption key. It obtains the key only after remote attestation succeeds and the deployment matches the expected manifest. Typical causes, each visible in the proxy log:

• active manifest does not match expected manifest: the deployment doesn't match the manifest the proxy expects. In the default mode, the proxy fetches the current manifest and retries. With a manifest pinned via --manifest-path, the proxy refuses to proceed until you update the file.
• An unmarshaling error while processing the manifest: the Privatemode API was updated with a manifest format this proxy version doesn't understand. Update the Privatemode proxy (the Docker image) to the latest version.
• fetching manifest: or a connection error: the proxy can't reach cdn.confidential.cloud or api.privatemode.ai. Check outbound network traffic and, if applicable, the TLS inspection settings.

Once the proxy holds a key, it refreshes it periodically. When the deployment changes, the refresh re-runs attestation against the new manifest. If a refresh fails, the proxy logs Failed to update secret and retries with a backoff of up to 60 seconds. Requests return status 500 until a refresh succeeds. The proxy never sends a request with a stale or unverified key.

The proxy exits at startup

When started with --api-key, the proxy runs attestation and the key exchange at startup. If that fails, it exits with an error starting with setting up secret manager configuration: trying API key:. The causes are the same as above, plus an invalid API key. Without --api-key, attestation runs on the first request that carries an API key, and a failure returns status 401 or 500 instead.

Readiness

GET /readyz returns 204 No Content once the proxy is listening. It reports that the process accepts connections, not that attestation has succeeded. To confirm attestation, check the proxy log for Secret updated successfully, or send a request.