Skip to content

Hook to make tracking time till first token (TTFT) easier #2373

@omar-scio

Description

@omar-scio

Title pretty much says it. Right now, to track the TTFT, you'd have to keep local state for this being the first LLM call that the Agent makes and have some custom logic. This is likely a metric many users would want to track, so having some hook on_first_token or something in the Agent seems generally useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions