Skip to content

Chat with a model hosted on Hugging Face Serverless Inference API

Source: R/provider-huggingface.R
chat_huggingface.Rd

Hugging Face hosts a variety of open-source and proprietary AI models available via their Inference API. To use the Hugging Face API, you must have an Access Token, which you can obtain from your Hugging Face account (ensure that at least "Make calls to Inference Providers" and "Make calls to your Inference Endpoints" is checked).

Built on top of chat_openai_compatible().

Known limitations

  • Some models do not support the chat interface or parts of it, for example google/gemma-2-2b-it does not support a system prompt. You will need to carefully choose the model.

Usage

chat_huggingface(
  system_prompt = NULL,
  params = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

Arguments

system_prompt

A system prompt to set the behavior of the assistant.

params

Common model parameters, usually created by params().

api_key

[Deprecated] Use credentials instead.

credentials

Override the default credentials. You generally should not need this argument; instead set the HUGGINGFACE_API_KEY environment variable. The best place to set this is in .Renviron, which you can easily edit by calling usethis::edit_r_environ().

If you do need additional control, this argument takes a zero-argument function that returns either a string (the API key), or a named list (added as additional headers to every request).

model

The model to use for the chat (defaults to "meta-llama/Llama-3.1-8B-Instruct"). We regularly update the default, so we strongly recommend explicitly specifying a model for anything other than casual use.

api_args

Named list of arbitrary extra arguments appended to the body of every chat API call. Combined with the body object generated by ellmer with modifyList().

echo

One of the following options:

  • none: don't emit any output (default when running in a function).

  • output: echo text and tool-calling output as it streams in (default when running at the console).

  • all: echo all input and output.

Note this only affects the chat() method.

api_headers

Named character vector of arbitrary extra headers appended to every chat API call.

Value

A Chat object.

See also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

if (FALSE) { # \dontrun{
chat <- chat_huggingface()
chat$chat("Tell me three jokes about statisticians")
} # }