Proxy configuration

The privatemode-proxy is a service that you must deploy to use the Privatemode API. Once started, the proxy serves as your API endpoint, handling all the heavy lifting to guarantee end-to-end encryption for you.

The privatemode-proxy does two things:

1. Verifies the Privatemode deployment at api.privatemode.ai. This is where your encrypted prompts are processed by the GenAI. The verification process is described in the attestation section.
2. Transparently encrypts user prompts and decrypts responses from the Privatemode API.

The privatemode-proxy is published as a Docker image on GitHub.

Running the container

The following command starts the privatemode-proxy and exposes it on host port 8080:

docker run -p 8080:8080 ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest

info

Supply chain security best practices recommend pinning containers by their hash. This means specifying the exact cryptographic digest of the container image, rather than relying on tags like latest or version labels. By doing so, you ensure that the exact, verified version of the container is used, which helps prevent issues like unexpected updates or potential compromise.

CLI flags

To see all available CLI option flags, use:

docker run ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest --help

Options

      --apiEndpoint string             The endpoint for the Privatemode API (default "api.privatemode.ai:443")
      --apiKey string                  The API key for the Privatemode API. If no key is set, the proxy will not authenticate with the API.
      --coordinatorEndpoint string     The endpoint for the Contrast coordinator. (default "coordinator.privatemode.ai:443")
      --coordinatorPolicyHash string   The hex-encoded hash of the policy to be enforced by the coordinator.
  -h, --help                           help for privatemode-proxy
  -l, --log-level string               set logging level (debug, info, warn, error, or a number) (default "info")
      --manifestPath string            The path for the manifest file. If not provided, the manifest will be read from the remote source.
      --port string                    The port on which the proxy listens for incoming API requests. (default "8080")
      --ssEndpoint string              The endpoint of the secret service. (default "secret.privatemode.ai:443")
      --tlsCertPath string             The path to the TLS certificate. If not provided, the server will start without TLS.
      --tlsKeyPath string              The path to the TLS key. If not provided, the server will start without TLS.
      --workspace string               The path into which the binary writes files. This includes the manifest log data in the 'manifests' subdirectory. (default ".")

Extract a static binary

If you want to run the proxy as a binary, you can extract it from the container image. Depending on your architecture (arm64 or amd64), insert the <arch> variable below to obtain a static Linux binary like this:

containerID=$(docker create --platform linux/<arch> ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest)
docker cp -L "${containerID}":/bin/privatemode-proxy ./privatemode-proxy
docker rm "${containerID}"

Outbound network traffic

When running the privatemode-proxy in an environment with restricted firewall settings, you might need to allow the following domains and ports:

• kdsintf.amd.com:443: for verifying the certificate chain of the signed CPU attestation report
• secret.privatemode.ai:443: for communication and verification of the secret service
• cdn.confidential.cloud:443: for fetching the latest manifest
• api.privatemode.ai:443: for sending encrypted LLM requests
• coordinator.privatemode.ai:443: for verifying the integrity of the deployment

Setting up TLS

If you run the privatemode-proxy on another machine and access it over a network, you must configure TLS to encrypt the network traffic.

Use the following flags to provide a TLS certificate and private key in PEM format:

• --tlsCertPath should point to the path of the TLS certificate file used to identify the server.
• --tlsKeyPath should point to the private key file that corresponds to the certificate.

By providing these, the privatemode-proxy will serve traffic from and to your application client via HTTPS, ensuring secure communication. If these flags aren't set, the privatemode-proxy will fall back to serving traffic over HTTP.

API key

The Privatemode API requires authentication with the API key you received when you signed up. You should use the --apiKey flag to provide it to the proxy and let it handle authentication.

Proxy updates

It’s possible that an update to the Privatemode API introduces a new manifest that's incompatible with your current version of the privatemode-proxy. In such cases, you may encounter issues where the updated manifest can't be processed by the privatemode-proxy. This is known as an "unmarshaling" error.

When this happens, please update the privatemode-proxy (the Docker image) to the latest version.

In the future, we will provide documentation on how to implement automatic updates, which will help mitigate these types of issues.

Manifest management

Whenever the privatemode-proxy verifies the Privatemode deployment, it relies on a manifest to determine whether the services should be trusted. The manifest contains fingerprints of expected configurations and states of trusted endpoints. If they differ from the actual configurations and states, the services aren't to be trusted.

By default, the manifest is managed automatically. Manual control requires extra work each time an update to the Privatemode API is rolled out.

Automatically

By default, the privatemode-proxy fetches a manifest from a file service managed by Edgeless Systems (you can get it here). Whenever validation of the Privatemode deployment fails, the privatemode-proxy fetches the latest manifest from the file service and retries validation. This allows the proxy to continue working without manual intervention, even if the deployment changes.

To ensure auditability of the enforced manifests over time, changes to the manifest are logged to the local file system. These logs serve as a transparency log, recording which manifest was used at what point in time to verify the Privatemode deployment.

The proxy writes a file called log.txt. For each manifest that's enforced by the proxy, log.txt contains a new line with the timestamp at which enforcement began and the filename of the manifest that was enforced.

log.txt and the corresponding manifests are stored in a folder manifests. You can use the CLI flag --workspace to control where the folder manifests is stored.

You should mount the workspace to the Docker host to ensure this transparency log isn't lost when the container is removed:

docker run -p 8080:8080 -v proxy-logs:/app/privatemode-proxy ghcr.io/edgelesssys/privatemode/privatemode-proxy:latest --workspace /app/privatemode-proxy

Manually

You can generate a manifest manually and provide the file path to the privatemode-proxy via its --manifestPath CLI flag.

warning

This approach isn’t recommended for production because updates to the Privatemode API are continuously rolled out. Each update includes a new manifest, which invalidates the current manifest and prevents successful validation through the privatemode-proxy. As a result, the manifest needs to be manually updated with each Privatemode API update.

Helm chart

You can use the privatemode-proxy Helm chart for easy deployment to Kubernetes.

Prerequisites

• Kubernetes 1.16+
• Helm 3+
• (Optional) Persistent Volume for workspace
• (Optional) ConfigMap for manifest file
• (Optional) TLS secret for certificates

Installation

You can install the chart with the following commands:

helm repo add edgeless https://helm.edgeless.systems/stable
helm repo update

helm install privatemode-proxy edgeless/privatemode-proxy

Configuration

API key

You should store the API key in a Kubernetes secret. Create it using:

kubectl create secret generic privatemode-api-key --from-literal=apiKey=your-api-key

Persistent volume

To persist the application’s data beyond the lifetime of the current deployment, you can configure a Persistent Volume. The data includes the transparency log and manifests that allow you to audit the enforced manifests over time.

First, create a PersistentVolumeClaim:

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: privatemode-proxy-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Then, configure these values for your chart:

config:
  workspace:
    enabled: true
    volumeClaimName: "privatemode-proxy-pvc"

TLS configuration

To enable TLS for communication between your application and the privatemode-proxy, provide the TLS certificate and key through a Kubernetes secret:

You can use cert-manager to manage the TLS secret. Or you can create it manually as follows:

kubectl create secret tls privatemode-proxy-tls \
  --cert=<path-to-cert> --key=<path-to-key>

Then, configure these values for your chart:

config:
  tls:
    enabled: true
    secretName: "privatemode-proxy-tls"

Manifest file configuration

While manually managing manifests isn't recommended (see Manifest management), you can pass in the manifest via a ConfigMap:

Create the ConfigMap from your manifest file:

kubectl create configmap privatemode-proxy-config --from-file=manifest.json=/path/to/your/manifest.json

Then, configure these values for your chart:

config:
  manifest:
    enabled: true
    configMapName: "privatemode-proxy-config"
    fileName: "manifest.json"
    mountPath: "/etc/config/manifest.json"

Accessing the proxy

Once the deployment is complete, you can configure your application to access the API through the privatemode-proxy service’s domain.

By default, the proxy can be accessed at the following URL:

http://privatemode-proxy-privatemode-proxy.default.svc.cluster.local:8080/v1

This URL is constructed as follows:

http://{helm-release}-privatemode-proxy.{namespace}.svc.cluster.local:{port}/v1

• {helm-release}: The name of your Helm release.
• {namespace}: The Kubernetes namespace where the proxy is deployed.
• {port}: The port configured for the proxy service (default is 8080).

If you configured a custom DNS entry in your cluster, adjust the URL accordingly. Replace the default service domain with your custom domain, ensuring that your application can correctly resolve and communicate with the privatemode-proxy service.

Uninstallation

You can uninstall the chart as follows:

helm uninstall privatemode-proxy