Skip to content

time_error is incompatible with sigma = read_exp_file _SD: every objective evaluation raises, and the fit silently returns inf #677

Description

@picoteepetunia

PyBNF 1.8.1 (bngsim 0.15.0, edition 2)

Summary

A time_error fit that sources its noise scale from a data column — sigma = read_exp_file _SD,
which ADR-0112 documents as supported — raises on every objective evaluation:

pybnf.printing.PybnfError: The following experimental data columns were not found
in the simulation output: {'Central_SD'}

MarginalizedTimeObjective inherits the strict _check_columns from SummationObjective,
which rejects any .exp column absent from the simulation output. It never picked up
LikelihoodObjective._check_columns, which is the one carrying the ADR-0021 exemption for
per-point _SD noise columns. So the noise column is treated as an unmatched observable.

Why this is worth fixing beyond the immediate error

The failure is silent and expensive. It does not surface at config load. It raises inside the
fit, per evaluation; PyBNF catches it, scores that parameter set inf, and continues. A 20-start
gntr run burns its full budget, prints Fitting complete, writes information criteria, and
records a best-fit line whose parameters are the geometric centre of every box with Obj = inf:

Completed 300 of 300 GNTR iterations
Stop criterion satisfied with objective function value of inf
Information criteria (best fit): AIC=4953.07  BIC=4961.7  AICc=4953.743593  (k=4, n=64, lnL=-2472.53)
Fitting complete
#	Simulation	Obj	Dose	ka	ke	sigma_t__FREE
	gntr_999	inf	100.0	3.1622776601683795	0.03162277660168379	5.0249999999999995

Nothing in the terminal output says the objective never evaluated even once. In my case that cost
9 minutes of wall clock before I went looking in the log.

Reproduction

Any edition-2 time_error fit whose .exp carries an _SD column:

noise_model = gaussian, sigma = read_exp_file _SD, time_error = truncated_normal, sigma_t = fit sigma_t__FREE
experiment: study, method: ode, data: data/study.exp, t_end: 264, n_steps: 1056

with data/study.exp:

# time Central Central_SD
216.5000 6.359659 3.783337
...

The class relationship can be checked without running a fit:

from pybnf.parse import load_config
from pybnf.objective import SummationObjective, LikelihoodObjective

obj = load_config('marginal.conf').obj
print(type(obj).__name__)                                            # MarginalizedTimeObjective
print([c.__name__ for c in type(obj).__mro__[:4]])
# ['MarginalizedTimeObjective', 'SummationObjective', 'ObjectiveFunction', 'object']
print(type(obj)._check_columns.__qualname__)                         # SummationObjective._check_columns
print(type(obj)._check_columns is SummationObjective._check_columns) # True
print(type(obj)._check_columns is LikelihoodObjective._check_columns)# False

obj._check_columns({'time', 'obs', 'obs_SD'}, {'time', 'obs'})
# PybnfError: The following experimental data columns were not found in the simulation output: {'obs_SD'}

Root cause

  • pybnf/measurement/time_error.py:275class MarginalizedTimeObjective(SummationObjective)
  • pybnf/measurement/time_error.py:318 — calls self._check_columns(exp_data.cols, compare_cols)
  • pybnf/objective.py:787SummationObjective._check_columns, which raises on any unmatched
    column
  • pybnf/objective.py:1929LikelihoodObjective._check_columns, which exempts each observable's
    data-column noise source (ADR-0021), and is not in the MRO

The omission looks like an oversight rather than a decision: MarginalizedTimeObjective did
carry over LikelihoodObjective._spec_for (time_error.py:584, whose docstring says it mirrors
LikelihoodObjective._spec_for), so the noise-spec plumbing was deliberately reused while the
column check that depends on it was not.

It also contradicts the ADR. ADR-0112 lists the supported noise scales as "constant / free / a
data column
(fix_at / fit / read_exp_file)", and read_exp_file is the one that cannot
work.

Suggested fix

Give MarginalizedTimeObjective the exempting check — either by hoisting
LikelihoodObjective._check_columns into a mixin both classes use, or by copying it the same way
_spec_for was copied. A regression test with an _SD column on a time_error fit would pin it.

Workaround

Declare the scale as a constant and drop the column: sigma = fix_at <value> with a two-column
.exp. That is only available when the noise scale genuinely is constant — a per-point _SD
column with time_error has no workaround.

Possibly affected elsewhere

Worth checking any shipped example or tutorial that pairs time_error with read_exp_file _SD;
lesson 49_measurement_time_uncertainty is the obvious one to look at.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions