Repository navigation
Conversation
Bun's fetch fails a request after 5 minutes with no data either way (BUN_CONFIG_HTTP_IDLE_TIMEOUT), in practice at 6 because its timer ticks once a minute. Bun sets process.versions.node, so modelFetch took the Node path, and Bun's built-in stand-in for undici has an empty Agent that ignores headersTimeout and bodyTimeout. A model that thought for more than 6 minutes failed on Bun, every retry the same. On Bun, modelFetch now calls Bun's fetch with `timeout: false`, which switches that wait off for the request. A timeout the caller passes is kept. Deno's fetch has no such wait and is unchanged. Checked on Bun 1.4.4 with the idle limit shrunk to 5 s: plain fetch failed at 12 s, while modelFetch and OpenAiCompatibleModelClient waited out a 30 s silence before the headers and in the body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#118 stopped Node's built-in fetch from cutting off a model that thinks for more than 300 s. Bun has the same kind of limit and #118 did not reach it: Bun's fetch fails a request after 5 minutes with no data either way (
BUN_CONFIG_HTTP_IDLE_TIMEOUT), in practice at 6 because its timer ticks once a minute. Bun setsprocess.versions.node, somodelFetchtook the Node path, and Bun's built-in stand-in for undici has an emptyAgentthat ignoresheadersTimeoutandbodyTimeout. A model that thought for more than 6 minutes failed on Bun, every retry the same way.What
modelFetchnow calls Bun's own fetch withtimeout: false, which switches that wait off for the request. Bun is checked before Node. Atimeoutthe caller passes is kept.fetchare unchanged. Deno's fetch has no such wait (20-minute silences got through on Deno 2.9.8).process.versions.bunBun sets and a recording global fetch). A CHANGELOG entry under Unreleased, Fixed.Checks
conformance:check.docs:api:checkin a plain clone: up to date.modelFetchandOpenAiCompatibleModelClientwaited out 8 minutes of silence before the headers and in the body.Related
Follow-up, not in this PR: stream model replies (
stream: true). Today the reply comes in one response, so the connection is silent for the whole generation and any idle limit on the way (a proxy, a gateway) can still cut it. With streaming, text flows as it is written and OpenRouter sends keep-alive comments while the model thinks.