Skip to content

Document missing functions/features and add multi-measure examples (from #30) #32

Description

@mgraffg

Context

#30 added multi-measure support to Perf/Difference (score_func/error_func as a list of
callables, mixed BiB directions, the .measure() factory attached to every metrics.py
wrapper). The implementation landed with tests and inline docstrings on the core
interface.py properties, but the documentation surface (Sphinx docs/source/*.rst via
automodule :members:, and the narrative walkthrough in quarto/CompStats.qmd) was not
updated to match, and a few pre-existing gaps remain uncovered.

Missing documentation found

  1. .measure() factory is invisible in the generated docs. Every wrapper in
    metrics.py (e.g. f1_score.measure = _f1_score_measure) attaches a .measure() factory
    used to build multi-measure Perf instances (see Perf's class docstring example in
    interface.py). metrics_docs() (utils.py), which injects the shared docstring for
    every wrapper, never mentions .measure(). Since .measure is a plain attribute on a
    function object (not a class method), Sphinx's automodule :members: in
    docs/source/metrics_api.rst never surfaces it either — this is the main entry point for
    Plan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together) #30's feature and it has no user-facing documentation anywhere.
  2. No narrative example of multi-measure usage. quarto/CompStats.qmd (compiled into the
    docs and referenced from docs/source/metrics_api.rst) only walks through the
    single-measure flow. There's no example showing score_func=[f1_score.measure(average='macro'), recall_score.measure(average='macro')], mixing score_func/error_func with different BiB
    directions, setting measure_names, or how .statistic/.se/.ci/.plot()/.difference()
    behave on a multi-measure Perf.
  3. macro_f1, macro_recall, macro_precision (added in Plan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together) #30) aren't mentioned in the
    narrative docs
    — only discoverable by browsing the auto-generated API reference.
  4. Minor pre-existing gaps, worth picking up in the same pass since they're directly
    documentation work:
    • Difference.sorting_func getter (interface.py:722) has no docstring.
    • utils.py helpers metrics_docs, dataframe, progress_bar have one-line/one-word
      stub docstrings that don't explain what they do or return.
    • Two comments in bootstrap.py (lines 56, 75) are written in Spanish
      (# Guardar el parámetro BiB, # Añadir BiB a los parámetros), which violates the
      English-only policy in CLAUDE.md.

Proposed scope

  • Update metrics_docs() in utils.py so the injected docstring for every metrics.py
    wrapper explains .measure() and links to Perf's multi-measure support.
  • Add docstrings for Difference.sorting_func and the thin utils.py helpers.
  • Translate the two stray Spanish comments in bootstrap.py to English.
  • Add a new section to quarto/CompStats.qmd (and regenerate/update whatever derived docs
    depend on it) demonstrating the multi-measure workflow end-to-end: composing two or more
    measures via .measure(), mixing score_func/error_func, custom measure_names,
    inspecting the resulting .statistic/.se/.ci, faceted .plot() output, and
    .difference().
  • Add a short mention of macro_f1/macro_recall/macro_precision as ready-made
    multi-measure-friendly convenience wrappers.

Non-goals

No behavior changes — this issue is documentation/examples only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions