A Chat is a sequence of user and assistant Turns sent
to a specific Provider. A Chat is a mutable R6 object that takes care of
managing the state associated with the chat; i.e. it records the messages
that you send to the server, and the messages that you receive back.
If you register a tool (i.e. an R function that the assistant can call on
your behalf), it also takes care of the tool loop.
You should generally not create this object yourself,
but instead call chat_openai() or friends instead.
Active bindings
conversation_idIdentifier for the current conversation. When set, it is recorded as the
gen_ai.conversation.idattribute on the OpenTelemetry spans emitted for subsequent model calls. AssignNULLto clear.Developer-facing: intended for frameworks that manage conversation history (e.g., Shiny apps). ellmer never generates an identifier on its own.
Methods
Chat$new()
Usage
Chat$new(provider, model = NULL, system_prompt = NULL, echo = "none")Arguments
providerA provider object.
modelA Model object.
system_promptSystem prompt to start the conversation with.
echoOne of the following options:
none: don't emit any output (default when running in a function).output: echo text and tool-calling output after the turn completes (default when running at the console).all: echo all input and output.
Console display occurs after a turn completes so ellmer can add citation markers and a source list to the response.
Note this only affects the
chat()method. You can override the default by setting theellmer_echooption.
Chat$get_turns()
Retrieve the turns that have been sent and received so far (optionally starting with the system prompt, if any).
Chat$get_rounds()
Retrieve the conversation grouped into Rounds. Each
Round pairs a user turn with the assistant and tool-result turns it
produced.
Chat$last_round()
The last Round of conversation. Note that system prompt
turns are included, equivalent to the last item in the list of rounds
returned by $get_rounds(include_system_prompt = TRUE).
Chat$set_model()
Update the model name. Note that unlike some of the
chat_*() functions, the model name is not validated against available
models for the provider.
Chat$get_tokens()
A data frame with token usage and cost data. There are four
columns: input, output, cached_input, and cost. There is one
row for each assistant turn, because token counts and costs are only
available when the API returns the assistant's response.
![[Deprecated]](figures/lifecycle-deprecated.svg)
Chat$token_count()
Estimate the token count for ... using the
provider's token counting endpoint.
Usage
Chat$token_count(..., include = c("new", "complete"), type = NULL)Arguments
...Input to count tokens for.
includeWhat to include in the count.
"new"counts tokens only for the contents of...."complete"estimates the total input tokens for the next request, including system prompt, tools, and conversation history.typeAn optional type specification for structured data extraction, created with a
type_()function.
Chat$file_upload()
![[Experimental]](figures/lifecycle-experimental.svg)
Upload a file to the chat's provider, once, so later turns can
reference it by id instead of re-sending its contents. Prefer this
over content_pdf_file(), content_image_file(), or
content_document_file() when a file is large or used across many
turns. Otherwise, sending the file inline is simpler: it isn't limited
to providers with a files API, and there's nothing stored on the
provider's side to expire or clean up.
File management is supported by chat_openai(), chat_anthropic(),
and chat_google_gemini(); other providers error. Provider notes:
Gemini files always expire after 48 hours (so
expires_in_hcan't be changed), and uploading waits until Gemini finishes processing the file (which can take a while for large video/audio), so the returned reference is always ready to use. The Files API isn't available on Vertex AI; there, upload the file to a Cloud Storage bucket and reference it withContentUploaded(uri = "gs://bucket/object", mime_type = ...).An OpenAI upload can also be referenced from a
chat_openai_compatible()chat pointed at OpenAI's Chat Completions API, except for images, which that API can't reference by id.
Arguments
pathPath to a file to upload.
mime_typeMIME type of the file. If not supplied, it's guessed from the file extension.
expires_in_hNumber of hours until the provider deletes the file. Defaults to 48. Anthropic accepts 1 to 2160 (90 days), OpenAI 1 to 720 (30 days), and both accept
Infto keep the file until you delete it yourself. Gemini always uses 48 and can't be changed.
Returns
A ContentUploaded that can be passed to $chat() and
friends in place of the file itself.
Chat$file_get()
Get a reference to a file previously uploaded to the chat's provider,
e.g. to reuse an upload from an earlier session. Use $file_list() to
find the id.
Arguments
idA file id string, or a ContentUploaded.
Returns
A ContentUploaded that can be passed to $chat() and
friends, with file metadata (filename, size_bytes, created_at,
expires_at, and any provider-specific fields) in its extra
property. OpenAI doesn't report a file's MIME type, so it's guessed
from the filename.
Chat$file_download()
Download a file from the chat's provider, writing it to path.
Note that providers only serve back model-generated files (e.g. batch
outputs); files you uploaded yourself can't be re-downloaded.
Arguments
idA file id string, or a ContentUploaded.
pathPath to write the downloaded file to.
Chat$file_delete()
Delete a file previously uploaded to the chat's provider.
Arguments
idA file id string, or a ContentUploaded.
Chat$last_turn()
The last turn returned by the assistant.
Usage
Chat$last_turn(role = c("assistant", "user", "system"))Chat$chat()
Submit input to the chatbot, and return the response as a simple string (probably Markdown).
Arguments
...The input to send to the chatbot. Can be strings or images (see
content_image_file()andcontent_image_url().echoWhether to emit the response to stdout as it is received. If
NULL, then the value ofechoset when the chat object was created will be used.
Chat$chat_structured()
Extract structured data.
Note: tool calling is disabled during structured data extraction. See
vignette("structured-data") for details and workarounds.
Arguments
...The input to send to the chatbot. This is typically the text you want to extract data from, but it can be omitted if the data is obvious from the existing conversation.
typeA type specification for the extracted data. Should be created with a
type_()function.echoWhether to emit the response to stdout as it is received. Set to "text" to stream JSON data as it's generated (not supported by all providers).
convertAutomatically convert from JSON lists to R data types using the schema. For example, this will turn arrays of objects into data frames and arrays of strings into a character vector.
Chat$chat_structured_async()
Extract structured data, asynchronously. Returns a promise that resolves to an object matching the type specification.
Arguments
...The input to send to the chatbot. Will typically include the phrase "extract structured data".
typeA type specification for the extracted data. Should be created with a
type_()function.echoWhether to emit the response to stdout as it is received. Set to "text" to stream JSON data as it's generated (not supported by all providers).
convertAutomatically convert from JSON lists to R data types using the schema. For example, this will turn arrays of objects into data frames and arrays of strings into a character vector.
Chat$chat_async()
Submit input to the chatbot, and receive a promise that resolves with the response all at once. Returns a promise that resolves to a string (probably Markdown).
Usage
Chat$chat_async(..., tool_mode = c("concurrent", "sequential"))Arguments
...The input to send to the chatbot. Can be strings or images.
tool_modeWhether tools should be invoked one-at-a-time (
"sequential") or concurrently ("concurrent"). Sequential mode is best for interactive applications, especially when a tool may involve an interactive user interface. Concurrent mode is the default and is best suited for automated scripts or non-interactive applications.
Chat$stream()
Submit input to the chatbot, returning streaming results. Returns A coro generator that yields strings. While iterating, the generator will block while waiting for more content from the chatbot.
Usage
Chat$stream(..., type = NULL, stream = c("text", "content"), controller = NULL)Arguments
...The input to send to the chatbot. Can be strings or images.
typeAn optional
type_()structured-data specification. When supplied, registered tools are suppressed and the completed assistant turn stores aContentJson. The provider constrains the response to JSON. Withstream = "text"(the default), structured stream chunks are raw JSON text; withstream = "content", they are Content objects. Streaming structured output requires native provider support; tool-based fallback is not supported.streamWhether the stream should yield only
"text"or ellmer's rich content types. Whenstream = "content",stream()yields Content objects.controllerAn optional
stream_controller()used to cancel the stream from outside the iteration loop.
Chat$stream_async()
Submit input to the chatbot, returning asynchronously streaming results. Returns a coro async generator that yields string promises.
Arguments
...The input to send to the chatbot. Can be strings or images.
typeAn optional
type_()structured-data specification. When supplied, registered tools are suppressed and the completed assistant turn stores aContentJson. The provider constrains the response to JSON. Withstream = "text"(the default), structured stream chunks are raw JSON text; withstream = "content", they are Content objects. Streaming structured output requires native provider support; tool-based fallback is not supported.tool_modeWhether tools should be invoked one-at-a-time (
"sequential") or concurrently ("concurrent"). Sequential mode is best for interactive applications, especially when a tool may involve an interactive user interface. Concurrent mode is the default and is best suited for automated scripts or non-interactive applications.streamWhether the stream should yield only
"text"or ellmer's rich content types. Whenstream = "content",stream()yields Content objects.controllerAn optional
stream_controller()used to cancel the stream from outside the iteration loop.
Chat$register_tool()
Register a tool (an R function) that the chatbot can use.
Learn more in vignette("tool-calling").
Arguments
toolA tool definition created by
tool().
Chat$register_tools()
Register a list of tools.
Learn more in vignette("tool-calling").
Arguments
toolsA list of tool definitions created by
tool().
Chat$set_tools()
Sets the available tools. For expert use only; most users
should use register_tool().
Arguments
toolsA list of tool definitions created with
tool().
Chat$on_tool_request()
Register a callback for a tool request event.
Chat$on_tool_result()
Register a callback for a tool result event.
Chat$on_request_start()
Register a callback that fires before each model request,
including each round of the tool loop. Use it to inspect the outgoing
request, or to compact the conversation with $set_turns().
turns includes the pending turn about to be sent, which $set_turns()
re-appends automatically. So compact with
chat$set_turns(compact(chat$get_turns())) rather than passing turns
back to $set_turns(), which would duplicate the pending turn.
Chat$on_request_end()
Register a callback that fires after each model request, before any tool calls in the response are executed. Use it to track latency or cost per request, or to observe tool requests before they run.
If the request is cancelled, turn is an AssistantPartialTurn with
NA tokens and cost. If the request errors, the callback does not fire.
Examples
chat <- chat_openai()
#> Using model = "gpt-5.6-terra".
chat$chat("Tell me a funny joke")
#> Why don’t skeletons fight each other?
#>
#> Because they don’t have the guts.