You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#30 added multi-measure support to Perf/Difference (score_func/error_func as a list of
callables, mixed BiB directions, the .measure() factory attached to every metrics.py
wrapper). The implementation landed with tests and inline docstrings on the core interface.py properties, but the documentation surface (Sphinx docs/source/*.rst via automodule :members:, and the narrative walkthrough in quarto/CompStats.qmd) was not
updated to match, and a few pre-existing gaps remain uncovered.
Missing documentation found
.measure() factory is invisible in the generated docs. Every wrapper in metrics.py (e.g. f1_score.measure = _f1_score_measure) attaches a .measure() factory
used to build multi-measure Perf instances (see Perf's class docstring example in interface.py). metrics_docs() (utils.py), which injects the shared docstring for
every wrapper, never mentions .measure(). Since .measure is a plain attribute on a
function object (not a class method), Sphinx's automodule :members: in docs/source/metrics_api.rst never surfaces it either — this is the main entry point for Plan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together) #30's feature and it has no user-facing documentation anywhere.
No narrative example of multi-measure usage.quarto/CompStats.qmd (compiled into the
docs and referenced from docs/source/metrics_api.rst) only walks through the
single-measure flow. There's no example showing score_func=[f1_score.measure(average='macro'), recall_score.measure(average='macro')], mixing score_func/error_func with different BiB
directions, setting measure_names, or how .statistic/.se/.ci/.plot()/.difference()
behave on a multi-measure Perf.
Minor pre-existing gaps, worth picking up in the same pass since they're directly
documentation work:
Difference.sorting_func getter (interface.py:722) has no docstring.
utils.py helpers metrics_docs, dataframe, progress_bar have one-line/one-word
stub docstrings that don't explain what they do or return.
Two comments in bootstrap.py (lines 56, 75) are written in Spanish
(# Guardar el parámetro BiB, # Añadir BiB a los parámetros), which violates the
English-only policy in CLAUDE.md.
Proposed scope
Update metrics_docs() in utils.py so the injected docstring for every metrics.py
wrapper explains .measure() and links to Perf's multi-measure support.
Add docstrings for Difference.sorting_func and the thin utils.py helpers.
Translate the two stray Spanish comments in bootstrap.py to English.
Add a new section to quarto/CompStats.qmd (and regenerate/update whatever derived docs
depend on it) demonstrating the multi-measure workflow end-to-end: composing two or more
measures via .measure(), mixing score_func/error_func, custom measure_names,
inspecting the resulting .statistic/.se/.ci, faceted .plot() output, and .difference().
Add a short mention of macro_f1/macro_recall/macro_precision as ready-made
multi-measure-friendly convenience wrappers.
Non-goals
No behavior changes — this issue is documentation/examples only.
Context
#30 added multi-measure support to
Perf/Difference(score_func/error_func as a list ofcallables, mixed BiB directions, the
.measure()factory attached to everymetrics.pywrapper). The implementation landed with tests and inline docstrings on the core
interface.pyproperties, but the documentation surface (Sphinxdocs/source/*.rstviaautomodule :members:, and the narrative walkthrough inquarto/CompStats.qmd) was notupdated to match, and a few pre-existing gaps remain uncovered.
Missing documentation found
.measure()factory is invisible in the generated docs. Every wrapper inmetrics.py(e.g.f1_score.measure = _f1_score_measure) attaches a.measure()factoryused to build multi-measure
Perfinstances (seePerf's class docstring example ininterface.py).metrics_docs()(utils.py), which injects the shared docstring forevery wrapper, never mentions
.measure(). Since.measureis a plain attribute on afunction object (not a class method), Sphinx's
automodule :members:indocs/source/metrics_api.rstnever surfaces it either — this is the main entry point forPlan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together) #30's feature and it has no user-facing documentation anywhere.
quarto/CompStats.qmd(compiled into thedocs and referenced from
docs/source/metrics_api.rst) only walks through thesingle-measure flow. There's no example showing
score_func=[f1_score.measure(average='macro'), recall_score.measure(average='macro')], mixingscore_func/error_funcwith differentBiBdirections, setting
measure_names, or how.statistic/.se/.ci/.plot()/.difference()behave on a multi-measure
Perf.macro_f1,macro_recall,macro_precision(added in Plan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together) #30) aren't mentioned in thenarrative docs — only discoverable by browsing the auto-generated API reference.
documentation work:
Difference.sorting_funcgetter (interface.py:722) has no docstring.utils.pyhelpersmetrics_docs,dataframe,progress_barhave one-line/one-wordstub docstrings that don't explain what they do or return.
bootstrap.py(lines 56, 75) are written in Spanish(
# Guardar el parámetro BiB,# Añadir BiB a los parámetros), which violates theEnglish-only policy in
CLAUDE.md.Proposed scope
metrics_docs()inutils.pyso the injected docstring for everymetrics.pywrapper explains
.measure()and links toPerf's multi-measure support.Difference.sorting_funcand the thinutils.pyhelpers.bootstrap.pyto English.quarto/CompStats.qmd(and regenerate/update whatever derived docsdepend on it) demonstrating the multi-measure workflow end-to-end: composing two or more
measures via
.measure(), mixingscore_func/error_func, custommeasure_names,inspecting the resulting
.statistic/.se/.ci, faceted.plot()output, and.difference().macro_f1/macro_recall/macro_precisionas ready-mademulti-measure-friendly convenience wrappers.
Non-goals
No behavior changes — this issue is documentation/examples only.