[bot] Hugging Face Transformers.js (`@huggingface/transformers`) not instrumented — no tracing for local text generation or embedding pipelines

[braintrust-bot]

Summary

Hugging Face publishes @huggingface/transformers on npm — a JavaScript/TypeScript library for running ML models locally via ONNX/WebGPU without any server or API key (v3.x / v4 released March 2025). This is distinct from @huggingface/inference (remote Hugging Face Inference API, already instrumented via js/src/auto-instrumentations/configs/huggingface.ts). The Transformers.js package runs text generation, embeddings, and other model pipelines in-process (Node.js, Bun, Deno, or the browser) and has zero instrumentation in this repository. Users who call @huggingface/transformers pipelines for text generation or embeddings get no Braintrust spans.

What instrumentation is missing

The @huggingface/transformers npm package exposes these execution surfaces, none of which are instrumented:

SDK Surface	Description
pipeline('text-generation', model)(prompt, options)	Local autoregressive text generation
pipeline('text2text-generation', model)(prompt, options)	Local seq2seq text generation (T5, BART, etc.)
pipeline('feature-extraction', model)(text, options)	Local text embeddings (pooled hidden states)
pipeline('question-answering', model)(question, context)	Local extractive QA
pipeline('summarization', model)(text, options)	Local text summarization
TextGenerationPipeline, FeatureExtractionPipeline, etc.	Individual pipeline classes with __call__()

The pipeline() factory is the primary API. Each call downloads (or loads from cache) a model, runs tokenization and inference locally, and returns the decoded output. The execution has clear start/end boundaries and meaningful inputs/outputs (prompt → generated text, or text → embedding vector) making it a strong candidate for span-level tracing.

No coverage in any instrumentation layer:

• No wrapper function (e.g. wrapTransformers())
• No diagnostics channels for Transformers.js pipeline execution
• No plugin handler in js/src/instrumentation/plugins/
• No auto-instrumentation config in js/src/auto-instrumentations/configs/
• No e2e test scenarios
• Grep for @huggingface/transformers or transformers.js across js/src/ returns zero matches

Distinction from existing @huggingface/inference instrumentation:

The existing huggingface.ts config patches @huggingface/inference, which makes HTTP calls to Hugging Face's hosted Inference API (requires an API key, network I/O). @huggingface/transformers runs models in-process via ONNX — no HTTP, no API key. The two packages have different module paths, different API shapes, and different execution models. A user who switches from @huggingface/inference to @huggingface/transformers (e.g. for local/offline inference) loses all tracing.

Braintrust docs status

not_found — Braintrust's Hugging Face integration page references @huggingface/inference (remote API) only. The Transformers.js package is not mentioned in any Braintrust docs.

• https://www.braintrust.dev/docs/integrations/ai-providers/hugging-face (covers @huggingface/inference only)

Upstream references

• @huggingface/transformers npm: https://www.npmjs.com/package/@huggingface/transformers
• Transformers.js v4 announcement: https://huggingface.co/blog/transformersjs-v4
• Official docs: https://huggingface.co/docs/transformers.js/index
• GitHub: https://github.com/huggingface/transformers.js
• Node.js server-side inference tutorial: https://huggingface.co/docs/transformers.js/en/tutorials/node

Local files inspected

• js/src/auto-instrumentations/configs/huggingface.ts — instruments @huggingface/inference only; module paths target @huggingface/inference dist files exclusively
• js/src/instrumentation/plugins/huggingface-plugin.ts — handles textGeneration, textGenerationStream, featureExtraction from @huggingface/inference; no @huggingface/transformers coverage
• js/src/instrumentation/plugins/huggingface-channels.ts — channels for @huggingface/inference only
• js/src/auto-instrumentations/configs/all.ts — @huggingface/transformers not listed
• e2e/scenarios/huggingface-instrumentation/ — tests cover @huggingface/inference only
• Full repo grep for @huggingface/transformers, transformers.js — zero matches