OpenAI-compatible
Chat completions
Use the Privatemode chat completions API to generate text from a prompt via a large language model. The API is compatible with the OpenAI Chat Completions API. To generate text, send your requests to the Privatemode proxy. Chat requests and responses are encrypted, both in transit and during processing.
Request body
modelstringrequired
The name of a currently available model. Note that models are updated regularly, and support for older models is discontinued over time. Use the models API to list available models.
messageslistrequired
The prompts for which a response is generated. Set a system prompt via a message with the system role to tailor the model's behavior to your needs.
Show properties
rolestringrequired
The role of the message author. One of system, user, or assistant.
contentstring or listrequired
The contents of the message.
streambooleandefault: false
Whether to stream the response as server-sent events. The SDKs return an iterator over the response chunks.
cache_saltstring
Opts this request into prompt caching: all requests that use the same salt share a cache, reducing response latency when the first part of a prompt is reused across requests. Keep salts private and use at least 256 bits of entropy, e.g. generated with openssl rand -base64 32. If caching is also configured in the Privatemode proxy, the value from the request takes precedence. With the OpenAI Python client, pass cache_salt via the extra_body argument.
response_formatobject
Enables structured outputs, following the OpenAI Chat Completions API.
When generating JSON output, always instruct the model to output JSON in your prompt. Otherwise, some models may emit endless whitespace until the token limit is hit. If that still happens, set frequency_penalty above 0.0 (for example 0.1) to discourage repetitive tokens, and set max_completion_tokens to avoid too long generations.
Additional parameters mirror the OpenAI API and are supported based on the model server's capabilities.
Returns
The response is a chat completion object. With stream set to true, the API instead returns a stream of chat completion chunk objects that carry the generated text in a delta field. The fields below describe the non-streaming response.
idstring
A unique identifier for the chat completion.
objectstring
Always "chat.completion", or "chat.completion.chunk" for streaming responses.
createdinteger
The Unix timestamp (in seconds) of when the chat completion was created.
modelstring
The model used.
choiceslist
The responses generated by the model.
Show properties
indexinteger
The index of the choice in the list.
messageobject
The message generated by the model.
Show properties
rolestring
Always "assistant".
contentstring
The generated text.
finish_reasonstring
The reason the model stopped generating, e.g. "stop".
usageobject
Token usage statistics.
Show properties
prompt_tokensinteger
The number of tokens in the prompt.
completion_tokensinteger
The number of generated tokens.
total_tokensinteger
The total number of tokens.
Other fields are consistent with the OpenAI API specifications.