darnit plan and darnit validate crash on the shipped openssf-baseline framework (reproduced on main 06e905e).
$ darnit plan
File ".../darnit/cli.py", line 425, in cmd_plan
for level in sorted(by_level.keys()):
TypeError: '<' not supported between instances of 'int' and 'NoneType'
$ darnit validate packages/darnit-baseline/src/darnit_baseline/openssf-baseline.toml
logger.info(f" By level: {', '.join(f'L{k}={v}' for k, v in sorted(by_level.items()))}")
TypeError: '<' not supported between instances of 'int' and 'NoneType'
Cause: both commands group controls by ControlConfig.level. openssf-baseline declares levels in tags.level (tags = { level = 1, ... }), so 65 of its 66 controls have level = None. One control, STAGE1-REF-SECURITY-01, sets the top-level level, so the keys mix int and None and sorted() raises. darnit plan -f community-spec works because none of its controls set level.
Same root cause, no crash: FrameworkConfig.get_controls_by_level(n) filters on ControlConfig.level, so it returns nothing for the 65 baseline controls that use tags.level. It has no production caller today. The audit path reads the level some other way and is not affected; a fix should check that.
Suggested direction:
- Give a control's level one source. Either
ControlConfig.level falls back to tags.level, or the shipped TOMLs move to the top-level field. The spec (framework-design.md) should say which one is canonical.
- Make
plan and validate sort with None last instead of crashing.
- Add a test that runs
plan and validate against every shipped framework TOML.
Found while removing dead code for #487 (#566).
Drafted with Claude Code; reviewed and posted by me.
darnit plananddarnit validatecrash on the shipped openssf-baseline framework (reproduced on main 06e905e).Cause: both commands group controls by
ControlConfig.level. openssf-baseline declares levels intags.level(tags = { level = 1, ... }), so 65 of its 66 controls havelevel = None. One control,STAGE1-REF-SECURITY-01, sets the top-levellevel, so the keys mixintandNoneandsorted()raises.darnit plan -f community-specworks because none of its controls setlevel.Same root cause, no crash:
FrameworkConfig.get_controls_by_level(n)filters onControlConfig.level, so it returns nothing for the 65 baseline controls that usetags.level. It has no production caller today. The audit path reads the level some other way and is not affected; a fix should check that.Suggested direction:
ControlConfig.levelfalls back totags.level, or the shipped TOMLs move to the top-level field. The spec (framework-design.md) should say which one is canonical.planandvalidatesort withNonelast instead of crashing.planandvalidateagainst every shipped framework TOML.Found while removing dead code for #487 (#566).
Drafted with Claude Code; reviewed and posted by me.