Skip to content

Raise ValueError/NotImplementedError instead of a generic RuntimeError based on DP error code#498

Draft
kalra-mohit wants to merge 1 commit into
OpenMined:devfrom
kalra-mohit:differentiate-status-code-errors
Draft

Raise ValueError/NotImplementedError instead of a generic RuntimeError based on DP error code#498
kalra-mohit wants to merge 1 commit into
OpenMined:devfrom
kalra-mohit:differentiate-status-code-errors

Conversation

@kalra-mohit

Copy link
Copy Markdown

Every result/build/merge call in algorithm_builder.hpp catches a non-OK absl::Status from the C++ DP library and turns it into std::runtime_error, which pybind11 maps to a plain Python RuntimeError. That's the same exception type whether you passed in a bad epsilon, asked for a result you'd already consumed, or hit a genuine internal failure in the library. From Python there's no way to except your way out of the cases that are actually your fault versus the ones that aren't, short of parsing the message string.

What I did

I traced through the vendored differential-privacy source at the pinned submodule commit to see what status codes these specific call sites (build, result, partial_result x3, merge) can actually produce. The two I could pin down cleanly:

  • kInvalidArgument - by far the most common. Covers things like bad bounds/overflow in the builder, and (somewhat non-obviously) calling result()/partial_result() a second time on the same algorithm instance, since Algorithm::PartialResult() in algorithm.h explicitly returns InvalidArgumentError once result_returned_ is already true.
  • kUnimplemented - returned by the base NoiseConfidenceInterval() for algorithms that don't override it.
  • kInternal - e.g. Count::Merge() on an empty Summary.
flowchart LR
    A[absl::Status from DP library] --> B{status.code}
    B -- kInvalidArgument --> C[py::value_error → ValueError]
    B -- kUnimplemented --> D[PyErr_SetString NotImplementedError]
    B -- everything else --> E[std::runtime_error → RuntimeError]
Loading

I added a small ThrowFromStatus() helper in a new status_errors.hpp and swapped it in at all six throw sites in algorithm_builder.hpp. kInvalidArgument maps to ValueError using pybind11's built-in py::value_error (already the pattern used for build errors in qunatile_tree.cpp, so this isn't introducing a new convention). kUnimplemented maps to NotImplementedError via PyErr_SetString, since pybind11 doesn't ship a wrapper for that one. Everything else - kInternal, kFailedPrecondition, anything I didn't verify - still raises RuntimeError, same as before. I didn't want to guess at mappings for codes I hadn't actually traced back to a real code path.

One existing test, test_bounded_mean.py::test_result_crash, asserted RuntimeError for the "call result() twice" case - that's actually the InvalidArgument path, so I updated it to expect ValueError and left a comment explaining why. I also added tests/algorithms/test_error_types.py with a couple of focused regression tests: one confirms the double-result case now raises ValueError (and explicitly isn't just accidentally a RuntimeError subclass), and another confirms merging an empty Summary still raises RuntimeError, so the change doesn't accidentally widen beyond what I verified.

Testing

This one's a C++ binding change, and I wasn't able to build the extension locally to run these against the real bindings - the CI workflow itself budgets 20 minutes just for the Google DP library build step, which is more than makes sense for a local sanity-check loop in this environment. What I did do:

  • Read the exact status-returning code paths in the pinned differential-privacy submodule commit for every scenario the new tests exercise, rather than guessing.
  • Ran black --check and clang-format --style=file (matching .clang-format, which is what this repo's CI uses) against every file I touched - all clean, no diffs.
  • py_compile on the Python test files.

I'm relying on CI here to actually build and run the test suite - opening this as a draft until that comes back green, at which point I'll flip it to ready for review.

Fixes #413

Every failure path in the algorithm bindings (build, result,
partial_result x3, merge) collapsed absl::Status errors from the C++ DP
library into a bare std::runtime_error, no matter what actually went
wrong. A bad epsilon and an internal library bug looked identical from
Python, so callers had no way to catch "you gave me a bad argument"
separately from "something broke internally" without string-matching
the exception message.

Traced through the vendored differential-privacy source (pinned
submodule commit) to see which status codes these call sites can
actually return: kInvalidArgument (bad bounds, calling result() twice,
etc.), kInternal, and kUnimplemented, with kInvalidArgument by far the
most common and the one users most need to distinguish. Added a small
ThrowFromStatus() helper that maps kInvalidArgument to ValueError (via
pybind11's existing py::value_error, already used elsewhere in this
codebase) and kUnimplemented to NotImplementedError, while leaving
everything else raising RuntimeError as before so this doesn't
silently reclassify errors we haven't verified.

One existing test (test_bounded_mean.py::test_result_crash) asserted
RuntimeError for calling result() twice, which is actually the
InvalidArgument case per algorithm.h's PartialResult() budget check -
updated it to expect ValueError, and added a dedicated test file that
exercises both the InvalidArgument -> ValueError path and confirms
Internal errors (e.g. merging an empty Summary) still raise
RuntimeError unchanged.

This repo's C++ extension isn't buildable in a lightweight way locally
(CI itself budgets 20 minutes just for the Google DP library build), so
I couldn't compile and run these against the real bindings here -
verified the status codes by reading the pinned upstream source instead
and I'm leaning on CI to confirm the build and test run.

Fixes OpenMined#413

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Raise different errors based on the error code returned from the DP Library

1 participant