We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 53ef884 commit 4364604Copy full SHA for 4364604
1 file changed
Tools/inspection/oracle_external_inspection.py
@@ -1,3 +1,20 @@
1
+"""Compare external inspection modes against a reference mode ("Oracle").
2
+
3
+This script reports the following validation metrics:
4
5
+- impossible: Number of stacks matching a known impossible pattern. The
6
+ classifiers are not exhaustive.
7
8
+- raw_tvd: Total variation distance between this mode's stack distribution
9
+ and the reference's distribution. Stacks classified as impossible are
10
+ excluded.
11
12
+- tvd_excess: raw_tvd minus tvd_floor.
13
14
+- tvd_floor: IID heuristic for the TVD expected from finite sampling of both
15
+ distributions. This is not a lower bound.
16
+"""
17
18
import argparse
19
import contextlib
20
import math
0 commit comments