These functions are used to prepare image URLs and files for input to the
chatbot. The content_image_url()
function is used to provide a URL to an
image, while content_image_file()
is used to provide the image data itself.
Usage
content_image_url(url, detail = c("auto", "low", "high"))
content_image_file(path, content_type = "auto", resize = "low")
content_image_plot(width = 768, height = 768)
Arguments
- url
The URL of the image to include in the chat input. Can be a
data:
URL or a regular URL. Valid image types are PNG, JPEG, WebP, and non-animated GIF.- detail
The detail setting for this image. Can be
"auto"
,"low"
, or"high"
.- path
The path to the image file to include in the chat input. Valid file extensions are
.png
,.jpeg
,.jpg
,.webp
, and (non-animated).gif
.- content_type
The content type of the image (e.g.
image/png
). If"auto"
, the content type is inferred from the file extension.- resize
If
"low"
, resize images to fit within 512x512. If"high"
, resize to fit within 2000x768 or 768x2000. (See the OpenAI docs for more on why these specific sizes are used.) If"none"
, do not resize.You can also pass a custom string to resize the image to a specific size, e.g.
"200x200"
to resize to 200x200 pixels while preserving aspect ratio. Append>
to resize only if the image is larger than the specified size, and!
to ignore aspect ratio (e.g."300x200>!"
).All values other than
none
require themagick
package.- width, height
Width and height in pixels.
Value
An input object suitable for including in the ...
parameter of
the chat()
, stream()
, chat_async()
, or stream_async()
methods.
Examples
chat <- chat_openai(echo = TRUE)
#> Using model = "gpt-4o".
chat$chat(
"What do you see in these images?",
content_image_url("https://www.r-project.org/Rlogo.png"),
content_image_file(system.file("httr2.png", package = "ellmer"))
)
#> The first image is the logo of the R programming language, which
#> consists of a gray letter "R" inside a gray oval.
#>
#> The second image is a hexagonal sticker for the "httr2" R package. It
#> features a stylized illustration of a baseball player swinging a bat
#> and the text "httr2" in a script font.
plot(waiting ~ eruptions, data = faithful)
chat <- chat_openai(echo = TRUE)
#> Using model = "gpt-4o".
chat$chat(
"Describe this plot in one paragraph, as suitable for inclusion in
alt-text. You should briefly describe the plot type, the axes, and
2-5 major visual patterns.",
content_image_plot()
)
#> This plot is a line graph depicting the sinusoidal functions sine
#> (sin) and cosine (cos) over the interval from 0 to 2π on the x-axis,
#> with values ranging from -1 to 1 on the y-axis. The sine curve, shown
#> in blue, begins at 0, peaks at 1 at π/2, crosses zero at π, reaches -1
#> at 3π/2, and returns to 0 at 2π. The cosine curve, in orange, starts
#> at 1, crosses zero at π/2, reaches -1 at π, returns to zero at 3π/2,
#> and finally returns to 1 at 2π. The two curves are out of phase by
#> π/2, showing complementary peaks and troughs.