> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Hugging Face

> Trace Hugging Face Inference, Transformers.js, and local Transformers pipeline calls in Braintrust to debug prompts, evaluate models, and monitor production usage

If you are a coding agent, prefer the Braintrust [`bt` CLI](/docs/reference/cli/quickstart) for repeatable, scriptable work: running evals, instrumenting code, querying logs, syncing data, managing functions, and configuring coding agents. Use the MCP server for reasoning over Braintrust data in conversation, and for capabilities the CLI doesn't cover, such as monitor views, alerts, and authoring evaluators, preprocessors, and facets.

[Hugging Face Inference](https://huggingface.co/docs/inference-providers/) provides a unified interface to LLMs, embeddings, and other models hosted on Hugging Face and routed providers. Braintrust traces chat completions, text generation, and feature extraction calls.

<View title="TypeScript" icon="https://img.logo.dev/typescriptlang.org?token=pk_BdcHD9e5SCW3j1rnJkNyMQ">
  <h2 id="setup-typescript">
    Setup
  </h2>

  Install the Braintrust and `@huggingface/inference` packages, then set your API keys. Requires `@huggingface/inference` v2.0.0 or later (any 2.x, 3.x, or 4.x release).

  <Steps>
    <Step title="Install packages">
      <CodeGroup>
        ```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        pnpm add braintrust @huggingface/inference
        ```

        ```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        npm install braintrust @huggingface/inference
        ```
      </CodeGroup>
    </Step>

    <Step title="Set environment variables">
      ```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      BRAINTRUST_API_KEY=<your-braintrust-api-key>
      HUGGINGFACE_API_KEY=<your-huggingface-token>

      # For organizations on the EU data plane, use https://api-eu.braintrust.dev
      # For self-hosted deployments, use your data plane URL
      # BRAINTRUST_API_URL=<your-braintrust-api-url>
      ```
    </Step>
  </Steps>

  <h2 id="auto-instrumentation-typescript">
    Auto-instrumentation
  </h2>

  To trace Hugging Face Inference SDK calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust's import hook to patch the SDK at runtime.

  <Steps>
    <Step title="Initialize Braintrust and call Hugging Face">
      <CodeGroup>
        ```javascript title="trace-huggingface-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        import { initLogger } from "braintrust";
        import * as huggingFace from "@huggingface/inference";

        initLogger({
          projectName: "huggingface-example",
          apiKey: process.env.BRAINTRUST_API_KEY,
        });

        const client = new huggingFace.InferenceClient(
          process.env.HUGGINGFACE_API_KEY,
        );

        const response = await client.chatCompletion({
          model: "meta-llama/Llama-3.1-8B-Instruct",
          provider: "featherless-ai",
          messages: [
            {
              role: "user",
              content: "Reply with exactly OK.",
            },
          ],
          max_tokens: 16,
          temperature: 0,
        });

        console.log(response.choices?.[0]?.message?.content);
        ```
      </CodeGroup>
    </Step>

    <Step title="Run with the import hook">
      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      node --import braintrust/hook.mjs trace-huggingface-auto.js
      ```

      The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects — compile your TypeScript to JavaScript, then run the compiled file with the import hook.

      <Note>
        If you're using a bundler, see [Trace LLM calls](/docs/instrument/trace-llm-calls#auto-instrumentation) for plugin and loader setup.
      </Note>
    </Step>
  </Steps>

  <h2 id="manual-instrumentation-typescript">
    Manual instrumentation
  </h2>

  To trace Hugging Face clients manually, wrap them yourself with `wrapHuggingFace()`. Use this when you want to trace selected clients, or wrap the module directly before constructing clients.

  <CodeGroup>
    ```javascript title="trace-huggingface-wrap.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    import { initLogger, wrapHuggingFace } from "braintrust";
    import * as huggingFace from "@huggingface/inference";

    initLogger({
      projectName: "huggingface-example",
      apiKey: process.env.BRAINTRUST_API_KEY,
    });

    const hf = wrapHuggingFace(huggingFace);
    const client = new hf.InferenceClient(process.env.HUGGINGFACE_API_KEY);

    const embedding = await client.featureExtraction({
      inputs: "Paris France",
      model: "thenlper/gte-large",
      provider: "hf-inference",
    });

    console.log(embedding);
    ```
  </CodeGroup>

  `wrapHuggingFace()` can wrap either:

  * The module import itself, including `InferenceClient`, `InferenceClientEndpoint`, `HfInference`, and `HfInferenceEndpoint`.
  * An already-constructed client instance.

  If you use routed or custom endpoints via `client.endpoint(...)`, Braintrust records the endpoint URL in span metadata.

  <h2 id="what-traced-typescript">
    What Braintrust traces
  </h2>

  Braintrust traces these `@huggingface/inference` SDK calls, capturing:

  * Chat completions, including streaming chat with first-token timing.
  * Text generation, including streaming text generation with first-token timing.
  * Feature extraction, summarized as embedding count, length, and batch count.
  * Token usage, including prompt, completion, and total tokens.
  * Request metadata, including model, provider, and selected request parameters.
  * Response identifiers, including ID, model, object, created, and finish reason.
  * Routed endpoint URL when calling `client.endpoint(...)`.

  <h2 id="resources-typescript">
    Resources
  </h2>

  * [Hugging Face Inference SDK](https://github.com/huggingface/huggingface.js/tree/main/packages/inference)
  * [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/)
  * [Trace LLM calls](/docs/instrument/trace-llm-calls)

  <h2 id="transformers-typescript">
    Transformers.js
  </h2>

  [Transformers.js](https://huggingface.co/docs/transformers.js/) (`@huggingface/transformers`) runs models locally in Node.js or the browser without a remote API call. Braintrust traces text generation, text-to-text generation, summarization, feature extraction, and question answering pipelines. Requires `@huggingface/transformers` v3.0.0 or later (any 3.x or 4.x release).

  <h3 id="setup-transformers-typescript">
    Setup
  </h3>

  Install Braintrust alongside `@huggingface/transformers`, then set your API key.

  <Steps>
    <Step title="Install packages">
      <CodeGroup>
        ```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        pnpm add braintrust @huggingface/transformers
        ```

        ```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        npm install braintrust @huggingface/transformers
        ```
      </CodeGroup>
    </Step>

    <Step title="Set environment variables">
      ```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      BRAINTRUST_API_KEY=<your-braintrust-api-key>

      # For organizations on the EU data plane, use https://api-eu.braintrust.dev
      # For self-hosted deployments, use your data plane URL
      # BRAINTRUST_API_URL=<your-braintrust-api-url>
      ```
    </Step>
  </Steps>

  <h3 id="auto-instrumentation-transformers-typescript">
    Auto-instrumentation
  </h3>

  To trace Transformers.js pipeline calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust's import hook to patch the SDK at runtime.

  <Steps>
    <Step title="Initialize Braintrust and call a pipeline">
      <CodeGroup>
        ```javascript title="trace-transformers-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
        import { initLogger } from "braintrust";
        import { pipeline } from "@huggingface/transformers";

        initLogger({
          projectName: "transformers-example", // Replace with your project name
          apiKey: process.env.BRAINTRUST_API_KEY,
        });

        const generator = await pipeline(
          "text-generation",
          "Xenova/distilgpt2",
        );

        const result = await generator("Hello, world!", { max_new_tokens: 20 });
        console.log(result);
        ```
      </CodeGroup>
    </Step>

    <Step title="Run with the import hook">
      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      node --import braintrust/hook.mjs trace-transformers-auto.js
      ```

      The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects — compile your TypeScript to JavaScript, then run the compiled file with the import hook.

      <Note>
        If you're using a bundler, see [Trace LLM calls](/docs/instrument/trace-llm-calls#auto-instrumentation) for plugin and loader setup.
      </Note>
    </Step>
  </Steps>

  <h3 id="manual-instrumentation-transformers-typescript">
    Manual instrumentation
  </h3>

  To trace Transformers.js calls manually, wrap the module or a pipeline instance with `wrapHuggingFaceTransformers()`.

  <CodeGroup>
    ```javascript title="trace-transformers-wrap-module.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    import { initLogger, wrapHuggingFaceTransformers } from "braintrust";
    import * as transformers from "@huggingface/transformers";

    initLogger({
      projectName: "transformers-example", // Replace with your project name
      apiKey: process.env.BRAINTRUST_API_KEY,
    });

    const tracedTransformers = wrapHuggingFaceTransformers(transformers);

    const generator = await tracedTransformers.pipeline(
      "text-generation",
      "Xenova/distilgpt2",
    );

    const result = await generator("Hello, world!", { max_new_tokens: 20 });
    console.log(result);
    ```

    ```javascript title="trace-transformers-wrap-pipeline.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    import { initLogger, wrapHuggingFaceTransformers } from "braintrust";
    import { pipeline } from "@huggingface/transformers";

    initLogger({
      projectName: "transformers-example", // Replace with your project name
      apiKey: process.env.BRAINTRUST_API_KEY,
    });

    const rawPipeline = await pipeline("text-generation", "Xenova/distilgpt2");
    const generator = wrapHuggingFaceTransformers(rawPipeline);

    const result = await generator("Hello, world!", { max_new_tokens: 20 });
    console.log(result);
    ```
  </CodeGroup>

  `wrapHuggingFaceTransformers()` can wrap either:

  * The module import itself, so all pipelines created from it are traced.
  * An already-constructed pipeline instance.

  <h3 id="what-traced-transformers-typescript">
    What Braintrust traces
  </h3>

  Braintrust traces these `@huggingface/transformers` pipeline calls, capturing:

  * Text generation (`text-generation`) calls, with the prompt as input and the generated text as output.
  * Text-to-text generation (`text2text-generation`) calls, with the input text and the generated text.
  * Summarization (`summarization`) calls, with the input passage and the summary.
  * Feature extraction (`feature-extraction`) calls, summarized as embedding count and embedding length.
  * Question answering (`question-answering`) calls, with context and question as input and the answer as output.
  * Request metadata, including model identifier and `provider: "huggingface"` on every span.

  <h3 id="resources-transformers-typescript">
    Resources
  </h3>

  * [Transformers.js documentation](https://huggingface.co/docs/transformers.js/)
  * [`@huggingface/transformers` on npm](https://www.npmjs.com/package/@huggingface/transformers)
  * [Trace LLM calls](/docs/instrument/trace-llm-calls)
</View>

<View title="Python" icon="https://img.logo.dev/python.org?token=pk_BdcHD9e5SCW3j1rnJkNyMQ">
  [Hugging Face Transformers](https://huggingface.co/docs/transformers/) runs models locally without a remote API call. Braintrust traces text generation, text-to-text generation, summarization, translation, feature extraction, and question answering pipelines. Requires `transformers` 4.42.0 or later.

  <h2 id="setup-python">
    Setup
  </h2>

  Install the Braintrust SDK and `transformers`, then set your API key.

  <Steps>
    <Step title="Install packages">
      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      pip install braintrust transformers torch
      ```

      <Note>
        Running Transformers pipelines requires a deep learning backend such as PyTorch (`torch`), TensorFlow, or JAX. Install at least one alongside `transformers`.
      </Note>
    </Step>

    <Step title="Set environment variables">
      ```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      BRAINTRUST_API_KEY=<your-braintrust-api-key>

      # For organizations on the EU data plane, use https://api-eu.braintrust.dev
      # For self-hosted deployments, use your data plane URL
      # BRAINTRUST_API_URL=<your-braintrust-api-url>
      ```
    </Step>
  </Steps>

  <h2 id="auto-instrumentation-python">
    Auto-instrumentation
  </h2>

  To trace Transformers pipeline calls without modifying your application code, call `auto_instrument()` once at startup. The `transformers` integration is enabled by default.

  <Steps>
    <Step title="Initialize Braintrust and call a pipeline">
      ```python title="trace_transformers_auto.py" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      import os

      import braintrust

      braintrust.auto_instrument()
      braintrust.init_logger(
          api_key=os.environ["BRAINTRUST_API_KEY"],
          project="transformers-example",  # Replace with your project name
      )

      from transformers import pipeline

      generator = pipeline("text-generation", model="gpt2")  # Replace with your model
      result = generator("Hello, world!", max_new_tokens=20)
      print(result)
      ```
    </Step>

    <Step title="Run your app">
      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      python trace_transformers_auto.py
      ```
    </Step>
  </Steps>

  <h2 id="manual-instrumentation-python">
    Manual instrumentation
  </h2>

  To trace specific pipelines manually, wrap them with `wrap_transformers()`. Wrapping applies to the pipeline class and is idempotent.

  ```python title="trace_transformers_wrap.py" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  import braintrust
  from braintrust.integrations.transformers import wrap_transformers
  from transformers import pipeline

  braintrust.init_logger(project="transformers-example")  # Replace with your project name

  generator = pipeline("text-generation", model="gpt2")  # Replace with your model
  generator = wrap_transformers(generator)

  result = generator("Hello, world!", max_new_tokens=20)
  print(result)
  ```

  `wrap_transformers()` accepts a pipeline instance or class. Unsupported pipeline types are returned unchanged.

  <h2 id="what-traced-python">
    What Braintrust traces
  </h2>

  Braintrust instruments local Transformers pipeline calls and creates an LLM-typed span per call:

  * Text generation (`text-generation`) calls, with the prompt as input and the generated text as output.
  * Text-to-text generation (`text2text-generation`) calls, with input text and the generated text.
  * Summarization (`summarization`) calls, with the input passage and the summary.
  * Translation calls (`translation` and language-specific variants such as `translation_en_to_fr`), with the source text and the translation.
  * Feature extraction (`feature-extraction`) calls, with output summarized as tensor shape.
  * Question answering (`question-answering`) calls, with context and question as input and the answer as output.
  * Request metadata including model identifier, task, device, dtype, and generation parameters (temperature, top\_p, max\_tokens, and more).
  * Streaming calls (those using a `streamer` argument) are not traced and pass through unchanged.

  <h2 id="resources-python">
    Resources
  </h2>

  * [Hugging Face Transformers documentation](https://huggingface.co/docs/transformers/)
  * [transformers on PyPI](https://pypi.org/project/transformers/)
  * [Trace LLM calls](/docs/instrument/trace-llm-calls)
</View>