Skip to content

fix: reranker sends untruncated code blocks → vLLM 400 'document is too long' #14

Description

@aider4ryder

Problem

When using a vLLM-hosted reranker (Qwen3-Reranker-8B), searches on repos with large functions fail:

Reranker API returned error status 400 Bad Request: {"detail":"document is too long"}

codeseek falls back to RRF results, losing reranker precision on exactly the cases where it matters most (large, complex functions).

Root cause

reranker_service.rs:79 sends each candidate's full code_block as a document:

let documents: Vec<String> = candidates.iter().map(|c| c.code_block.clone()).collect();

No truncation is applied. When a code block exceeds the reranker model's token limit, vLLM rejects the entire batch.

Proposed fix

Truncate each document to a configurable max length (e.g. 512 chars / ~128 tokens) before sending:

let max_doc_len = self.config.max_doc_length.unwrap_or(512);
let documents: Vec<String> = candidates
    .iter()
    .map(|c| {
        if c.code_block.len() > max_doc_len {
            format!("{}...", &c.code_block[..max_doc_len])
        } else {
            c.code_block.clone()
        }
    })
    .collect();

For reranking, the function signature + first few lines are usually sufficient to judge relevance. Full source is available via codeseek_snippet after ranking.

Environment

  • codeseek 0.1.30
  • vLLM with Qwen3-Reranker-8B (max_length ~8192 tokens, but batch of 15-20 code blocks can exceed this)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions