Chat with an OpenAI-compatible model

This function is for use with OpenAI-compatible APIs, also known as the chat completions API. If you want to use OpenAI itself, we recommend chat_openai(), which uses the newer responses API.

Many providers offer OpenAI-compatible APIs, including:

• Ollama for local models

• vLLM for self-hosted models

• Various cloud providers with OpenAI-compatible endpoints

Usage

chat_openai_compatible(
  base_url,
  name = "OpenAI-compatible",
  system_prompt = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  api_headers = character(),
  preserve_thinking = FALSE,
  echo = c("none", "output", "all")
)

Arguments

base_url

The base URL to the endpoint. This parameter is required since there is no default for OpenAI-compatible APIs.

name

The name of the provider; this is shown in token_usage() and is used to compute costs.

system_prompt

A system prompt to set the behavior of the assistant.

api_key

Use credentials instead.

credentials

Credentials to use for authentication. If not provided, will attempt to use the OPENAI_API_KEY environment variable.

model

The model to use for chat. No default; depends on your provider.

params

Common model parameters, usually created by params().

api_args

Named list of arbitrary extra arguments appended to the body of every chat API call. Combined with the body object generated by ellmer with modifyList().

api_headers

Named character vector of arbitrary extra headers appended to every chat API call.

preserve_thinking

If TRUE, reasoning content returned by the model is included when sending conversation history back to the API. If FALSE (the default), reasoning content is still captured in the turn but dropped from subsequent requests. Set to TRUE if your provider requires or benefits from seeing prior reasoning in multi-turn conversations.

echo

One of the following options:

• none: don't emit any output (default when running in a function).

• output: echo text and tool-calling output as it streams in (default when running at the console).

• all: echo all input and output.

Note this only affects the chat() method.

Value

A Chat object.

See also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

if (FALSE) { # \dontrun{
# Example with Ollama (requires Ollama running locally)
chat <- chat_openai_compatible(
  base_url = "http://localhost:11434/v1",
  model = "llama2"
)
chat$chat("What is the difference between a tibble and a data frame?")
} # }