Skip to content

RFC: external configuration selection around extension/llm — is any upstream example needed? #22761

Description

@ot4562-glitch

Hi maintainers — updating this RFC because the downstream project direction has narrowed materially since the original proposal.

ExactScope v1.2 is now under active development as a cold selector/exporter, not an application-owned runtime policy/finalizer layer. The current goal is to consume measurements produced elsewhere, select a minimum-sufficient configuration only when comparative evidence supports it, emit the host's native settings, and then have no ExactScope dependency in the production path.

For ExecuTorch specifically, the current extension/llm public surface may already be sufficient. GenerationConfig exposes native settings such as:

  • grammar / grammar_type;
  • max_new_tokens;
  • seq_len;
  • temperature;
  • other generation behavior owned by ExecuTorch.

That changes the upstream question. I am not asking ExecuTorch to add ExactScope semantics, an inference wrapper, RAG, a finalizer, or a new execution path. If an external tool can simply choose values for the existing GenerationConfig and hand them to the normal runner, then the preferred ExactScope v1.2 result is actually no ExecuTorch runtime change at all.

The remaining question is only whether a small generic example or documentation addition would be useful to ExecuTorch users for this pattern:

externally measured candidate settings
        -> choose one offline
        -> ordinary GenerationConfig
        -> existing IRunner/TextLLMRunner execution

If the existing docs/examples already cover this adequately, I am happy to close this proposal rather than manufacture an integration.

Current downstream evidence remains intentionally limited: earlier adapter/fake-runner work compiled against the public IRunner surface, but a real .pte + tokenizer + TextLLMRunner smoke is still pending, so I am not claiming real-model ExecuTorch compatibility.

ExactScope repository: https://github.com/ot4562-glitch/ExactScope

The latest stable ExactScope release remains v1.1.0; v1.2 is active development/research. v1.2.0-alpha.1 is a frozen engineering checkpoint, not a stable release or a cross-runtime compatibility claim.

Would you prefer that this RFC simply close as already-covered by the existing GenerationConfig/LLM docs, or is there a small generic example/documentation gap here that would be useful to contribute?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

rfcRequest for comment and feedback on a post, proposal, etc.

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions