fix(claude): stop counting base64 attachments as raw characters in token estimates

[DevMello]

Summary

• count_tokens estimated base64 attachments as raw characters: a 2MB pasted screenshot reported ~680k input tokens instead of a real cost around 1.6k.
• Attachment sources are now priced per attachment (sniffed pixel dimensions / 750, else decoded bytes / 512, min 256) at both the count_tokens endpoint and the cursor/kiro usage-log floor. Text-only bodies estimate exactly as before.

Verification

• bun run typecheck && bun run lint:gui && bun run test && bun run privacy:scan
• 5 new tests: endpoint bound for a 512KB image, PNG dimension pricing, tool_result nesting, text-only and text-document parity.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Summary by CodeRabbit

• Improvements

  • Improved token estimates for image and document attachments, including base64-encoded content.
  • Image dimensions and document types now contribute more accurately to token counts.
  • Usage reporting and token-count results are more consistent for multimodal requests.
  • Text-only requests continue to use existing token estimation behavior.

• Bug Fixes

  • Prevented encoded attachments from inflating token counts based solely on encoded length.
  • Corrected handling of base64 padding and nested document attachments.
  • Preserved accurate counting for text content, tool inputs, and schemas.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f1615f2e-01db-4c5a-9cdd-863d2a605e44

📥 Commits

Reviewing files that changed from the base of the PR and between 791e0fb and efe0a10.

📒 Files selected for processing (2)

• src/server/claude-messages.ts
• tests/claude-messages-endpoint.test.ts

---

📝 Walkthrough

Walkthrough

Claude request token estimation now treats base64 image and document attachments as bounded data. It uses image dimensions or decoded byte size. Ordinary text remains character-estimated. Routed usage logging and /count_tokens use the new estimator.

Changes

Claude token estimation

Layer / File(s)	Summary
Attachment-aware estimator and integrations src/server/claude-messages.ts	estimateClaudeRequestTokens separates base64 attachments from serialized text. Image attachments use sniffImageDimensions; other attachments use decoded byte size. Nested tool_result.content is handled recursively. Routed usage logging and /count_tokens use the estimator.
Estimator behavior coverage tests/claude-messages-endpoint.test.ts	Tests cover bounded image estimates, dimension-based pricing, nested PDF data, base64 padding, tool_use.input, tool schemas, text documents, and text-only parity with estimateTokens. A helper creates dimensioned PNG data.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CountTokens as /count_tokens
  participant UsageLogging as Routed usage logging
  participant Estimator as estimateClaudeRequestTokens
  participant ImageSniffer as sniffImageDimensions
  participant TextEstimator as estimateTokens
  CountTokens->>Estimator: request content and modelId
  UsageLogging->>Estimator: system, messages, tools, and modelId
  Estimator->>ImageSniffer: inspect base64 image data
  ImageSniffer-->>Estimator: image dimensions
  Estimator->>TextEstimator: estimate text and adjusted content
  TextEstimator-->>Estimator: token estimate
  Estimator-->>CountTokens: estimated token count
  Estimator-->>UsageLogging: estimated token count

Loading

Possibly related PRs

• lidge-jun/opencodex#939: Both changes modify Claude document attachment handling, including nested document content.
• lidge-jun/opencodex#953: Both changes modify Claude message handling and attachment-related token processing.

Suggested reviewers: wibias, ingwannu, lidge-jun

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: excluding base64 attachments from raw-character token estimates.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/server/claude-messages.ts`:
- Line 869: Update the decoded-size calculation in the attachment token
estimator to remove trailing base64 padding characters before converting encoded
length to bytes, so a padded 131,072-byte payload estimates 256 tokens. Add a
focused regression test alongside the existing attachment-related tests covering
that exact padded payload and expected token count.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3c7b18fa-fd1d-4eac-a461-619c523d1e12

📥 Commits

Reviewing files that changed from the base of the PR and between 5d2973c and a6dc37a.

📒 Files selected for processing (2)

• src/server/claude-messages.ts
• tests/claude-messages-endpoint.test.ts

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6dc37aee0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/server/claude-messages.ts`:
- Around line 892-902: Restrict the attachment-processing logic around the
visible image/document branch to Claude protocol content-block containers,
including nested tool_result.content, and do not recurse into arbitrary
tool_use.input values. Preserve attachment token estimation and data redaction
only for genuine content blocks. Add a focused regression test alongside the
existing claude-messages tests using a complete attachment-shaped object in
tool_use.input and assert it uses the normal serialized-text estimate.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d5dd6c5c-bb5c-4215-976e-73d0bf1bcd15

📥 Commits

Reviewing files that changed from the base of the PR and between a6dc37a and 3b78fdd.

📒 Files selected for processing (2)

• src/server/claude-messages.ts
• tests/claude-messages-endpoint.test.ts

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3b78fddf6b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Wibias]

[GD] Verdict: changes-requested

TLDR

• PR: #983 — fix(claude): stop counting base64 attachments as raw characters in token estimates
• Head: 1ad2be0 on dev (mergeStateStatus: CLEAN)
• Decision: Useful and correct on the code; needs owner actions — resolve the two verified-addressed Codex-connector threads and consider the bounded simplification candidates
• Usefulness: Fixes a real estimation bug (2MB screenshot counted as ~680k tokens instead of ~1.6k)
• Bugs: none blocking
• Security: none confirmed
• Spec / standards: clean — matches the claimed behavior, tests prove it
• Reviews: 2 Codex-connector threads verified-addressed with in-thread replies; 2 CodeRabbit threads resolved; owner should resolve the 2 remaining open threads
• Base / CI: green on 1ad2be0; local typecheck/tests/privacy green
• Gate: none
• Owner actions (foreign PR): resolve the 2 open Codex-connector threads (verified addressed); consider 3 bounded simplification candidates (list in full verdict)
• Bottom line: Ship after the owner resolves the two addressed-but-open Codex threads and reviews the simplification candidates; no blocking code defects found.

Full verdict

Semantic propagation

• Concepts audited: base64 attachment token estimation; text-only estimate parity; tool_use.input/tool-schema preservation
• Authoritative sources: src/adapters/anthropic-image-guard.ts (sniffImageDimensions), src/adapters/kiro.ts (estimateKiroImageTokens), src/lib/token-estimate.ts (estimateTokens), src/claude/inbound.ts (translator semantics)
• Producers and consumers checked: handleClaudeMessagesWithBudget usage-log floor and handleClaudeCountTokens endpoint both route through estimateClaudeRequestTokens; request-log finalizedUsage applies max(reported, estimate)
• Public/derived representations checked: /v1/messages/count_tokens response; request-log usageLogInputTokens; Kiro adapter image estimator (same 750/512 + min-256 shape)
• Material variant partitions checked: image vs document; sniffable dimensions vs decoded-bytes fallback; tool_result.content nesting; tool_use.input; tool schemas; text-source documents; text-only parity; base64 padding
• Positive and negative assertions checked: each partition has a focused test; negative tests prove tool_use.input and schemas are NOT stripped
• Unmapped surfaces: none
• Unproven equivalence assumptions: none
• Representation mismatches: none
• Variant coverage gaps: none
• Axis verdict: pass

Linked: none

Usefulness

Fixes a real correctness bug in token estimation: a 2MB pasted screenshot (~2.7M base64 chars) was previously counted as ~680k input tokens by the plain char/4 estimator, whereas the actual Anthropic cost is ~pixels/750 ≈ 1.6k tokens. The PR prices base64 attachments per-attachment at both the /count_tokens endpoint and the cursor/kiro usage-log floor, and preserves the prior text-only estimate exactly. This is genuinely useful and the claimed value is delivered.

Bugs / correctness

• Method: bug-review.md — Bugbot: n/a (no Cursor Bugbot this session; complementary pass run); static: run (typecheck + focused + full suite); trio: done (Finder/Challenger/Arbiter over the diff); complementary: done (all lenses incl. input-shape, boundary, parsing/serialization, error-propagation, concurrency, lifecycle)
• Findings: none blocking. Verified: base64 padding handling (unpadded length), pixel-dimension pricing, decoded-bytes fallback, tool_result.content recursion bounded, tool_use.input/schemas not stripped, text-only parity preserved, min-256 floor, Math.max(1, …) guarantee
• Fixed this session: none (foreign PR; no edits)

Security

• Scope reviewed: authn, secrets/config, AI/agent/MCP, IaC/Docker, data storage, authz, injection — no credential transport, no OAuth, no token handling, no new outbound requests, no secrets/logging exposure
• Findings: none confirmed. The estimator never logs or emits request bodies; it only sums integers. No new attack surface.
• Fixed this session: none

Spec / standards

• Spec source: PR body (summary + verification)
• Gaps: none — the implementation matches the claimed behavior, the tests assert the partitions, and no unjustified scope added. Standards: new helper is small, focused, typed; follows existing estimateTokens/sniffImageDimensions conventions; no repo-rule violations.

Reviews

• Owners/maintainers: none open
• Bots (CodeRabbit/Codex/Bugbot): CodeRabbit threads resolved (base64 padding + content-block restriction, both addressed in commits 3b78fdd/1ad2be0); Codex-connector threads (restrict stripping + preserve attachment-shaped tool-use args) verified-addressed at 1ad2be0 with [GD] in-thread replies and matching regression tests. The 2 Codex threads remain open (not resolvable from this review account); owner should resolve them.

Base / CI

• Behind/conflicts: head is 3 ahead / 5 behind base 5d2973c; mergeStateStatus CLEAN; the 5 base commits touch only anthropic.ts/inbound.ts thinking logic, disjoint from this diff
• Required checks: green on 1ad2be0 (all test 1-4/4, macos, gates, ci, keyring *, npm-global *, enforce-target, label, changes, windows skipped)
• Local tip compile/tests: bun run typecheck pass; bun test tests/claude-messages-endpoint.test.ts 36 pass; bun run privacy:scan pass; bun run lint:gui pass; full bun run test 8035 pass / 14 fail — all 14 failures are Windows symlink EPERM privilege failures in unrelated files (config.test.ts, native-main, responses state, real-home write guard), present on base too, not PR regressions

Simplification (for the PR owner)

Bounded, behavior-preserving candidates only; nothing was edited or pushed (foreign PR):

1. src/server/claude-messages.ts:862 — remove the stale dangling comment /** Documented approximation: serialize system+messages+tools, run the char estimator. */; it sits directly above the new estimator doc comment and no longer describes the code. Risk: none (comment only). Validation: typecheck.
2. tests/claude-messages-endpoint.test.ts:941-949 — the new countTokensPngBase64 helper is a near-duplicate of pngBase64/pngHeaderBytes in tests/anthropic-image-guard.test.ts. Optionally reuse/import that helper instead of re-defining the PNG header. Risk: low (test-only); validation: run both test files. Not required — the duplicate is small and the files are independent suites.
3. src/server/claude-messages.ts:648 — anthropicBody as Rec at the usage-log call site is a wider cast than the structural parameter { system?; messages?; tools? }. Optionally narrow to the structural type or reuse the endpoint's raw as Rec pattern. Risk: none; validation: typecheck.

These are optional; candidates 2 and 3 are stylistic and can be skipped without loss.

Gate

none

Bottom line

The PR is correct, useful, and well-tested. It fixes a real estimation bug, keeps text-only parity, and the scoping fix (1ad2be0) addresses both Codex-connector findings with regression tests. The only remaining items are owner-side: resolve the two verified-addressed Codex threads (they stay open from this review account), and optionally apply the bounded simplification candidates. No blocking code, security, or spec defects.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.