feat: Decouple compare_frames tolerance defaults from diffly.testing

[Hayden Anderson (haydena7)]

Closes #59.

Motivation

compare_frames, the CLI, and diffly.testing shared one pair of tolerance defaults (abs_tol=1e-08, rel_tol=1e-05), inherited from polars.testing.assert_frame_equal. Those defaults make sense for a test assertion that aims to be a drop-in replacement, but a diff report should bias toward flagging: with rel_tol=1e-05, a $1 change on a $100,000 value is hidden.

One correction to the issue's framing: the reason diffly.testing keeps the looser pair is not a general claim that assertions should tolerate more but rather parity with polars.testing.assert_frame_equal, so adopting diffly doesn't change which tests pass.

Changes

Behavior change for compare_frames and the CLI: defaults are now abs_tol=0.0, rel_tol=1e-09, matching polars.Expr.is_close and math.isclose (PEP 485). diffly.testing is unchanged.

• _utils.py: split ABS_TOL_DEFAULT/REL_TOL_DEFAULT into *_COMPARE_DEFAULT and *_TESTING_DEFAULT; call sites updated.
• cli.py: help strings no longer hardcode the old values.
• Tests: parametrized test asserting compare_frames flags what assert_frame_equal tolerates (1.0 vs 1.0 + 1e-7, 0.0 vs 1e-17).
• Docs: tolerances guide rewritten around the two default sets, with a note on near-zero behavior under abs_tol=0.0; testing guide cross-references it.

Upgrade note: users with near-zero values that differ by cancellation noise will see them flagged; set abs_tol for the data's scale.

[Copilot]

🟡 Changes recommended

The default-tolerance behavior split is a user-visible change and needs a targeted regression test to lock in the intended defaults and the documented divergence between compare_frames and diffly.testing.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates Diffly’s default floating-point tolerances to better suit diff reporting by decoupling compare_frames/CLI defaults (stricter, abs_tol=0.0, rel_tol=1e-09) from diffly.testing defaults (kept looser for polars.testing.assert_frame_equal parity).

Changes:

• Split tolerance constants into comparison vs testing defaults and update call sites accordingly.
• Update CLI option defaults/help text to reflect the new comparison defaults.
• Rewrite tolerance documentation to explain the two default sets and the near-zero behavior with abs_tol=0.0.

File summaries

File	Description
diffly/_utils.py	Introduces separate constants for comparison vs testing default tolerances.
diffly/comparison.py	Switches compare_frames default tolerances to the new comparison defaults.
diffly/_conditions.py	Aligns internal float comparison expression defaults with the comparison defaults.
diffly/testing.py	Keeps testing APIs on the looser defaults via the new testing constants.
diffly/cli.py	Updates CLI defaults/imports and help text to match the new comparison defaults.
docs/guides/features/tolerances.ipynb	Updates tolerances guide to document the new defaults and their implications.
docs/guides/features/testing.md	Clarifies that diffly.testing defaults match polars.testing and differ from compare_frames.
tests/test_performance.py	Updates performance test imports/usages to the new comparison default constant names.

Review details

• Files reviewed: 8/8 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Oliver Borchert (borchero)]

LGTM, thanks!

[Oliver Borchert (borchero)]

Hayden Anderson (@haydena7) could you fix the merge conflict?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (707b796) to head (6c35e47).

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #64   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           13        13           
  Lines         1138      1140    +2     
=========================================
+ Hits          1138      1140    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.