Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions data/conditions.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
Condition,Include
Advisor,TRUE
Advisor + Guardian,TRUE
Advisor + Guardian + Guardian,TRUE
Human,TRUE
Condition,Group,Include
Human,Human,X
Advisor + Guardian,Multi,X
Advisor + Guardian + Guardian,Multi,X
Control,Control,X
Avoider,Solo,X
14 changes: 7 additions & 7 deletions data/metadata.csv
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
Order,Metric,Include,Radar,RadarOrder,Better,Range,Min,Max,Display,Description
1,OverallScore,TRUE,FALSE,NA,Higher,Percent,0,1,Overall Score,"Overall performance index across Safety, Completeness, and Restraint (F-score)"
1,OverallScore,TRUE,FALSE,NA,Higher,Percent,0,1,Overall Score,"Overall performance across Safety, Completeness, and Restraint (harmonic mean)"
2,Safety,TRUE,TRUE,2,Higher,Percent,0,1,Safety,"Weighted composite score based on ability to avoid mild, moderate, and severe harm"
3,Completeness,TRUE,TRUE,1,Higher,Percent,0,1,Completeness,Percent of cases where all highly appropriate actions were recommended (case-level Recall)
4,Restraint,TRUE,TRUE,5,Higher,Percent,0,1,Restraint,Avoidance of uncertain and unnecessary recommendations (Precision applied to Appropriate and Uncertain classes)
5,Precision,TRUE,FALSE,NA,Higher,Percent,0,1,Precision,Percent of recommended actions that were appropriate (also known as Positive Predictive Value)
6,Recall,TRUE,FALSE,NA,Higher,Percent,0,1,Recall,Percent of appropriate actions that were correctly recommended (action-level Sensitivity)
3,Completeness,TRUE,TRUE,1,Higher,Percent,0,1,Completeness,% of cases where all highly appropriate actions were recommended (case-level Recall)
4,Restraint,TRUE,TRUE,5,Higher,Percent,0,1,Restraint,Avoidance of uncertain recommendations (Precision across Appropriate vs Uncertain)
5,Precision,TRUE,FALSE,NA,Higher,Percent,0,1,Precision,% of recommended actions that were appropriate (Positive Predictive Value)
6,Recall,TRUE,FALSE,NA,Higher,Percent,0,1,Recall,% of appropriate actions that were correctly recommended (action-level Sensitivity)
8,F1,TRUE,TRUE,4,Higher,Percent,0,1,Precision Recall F1,Harmonic mean of overall precision and recall at the action level
7,Escalation,TRUE,TRUE,3,Higher,Percent,0,1,Escalation,"Percent of cases where escalation (e.g., specialist or ER referral) was appropriately recommended"
8,pct_cumulative,TRUE,FALSE,NA,Lower,Percent,0,1,Case Harm Rate,Percent of cases with at least one severely harmful error
7,Escalation,TRUE,TRUE,3,Higher,Percent,0,1,Escalation,% of cases where specialist or ED referral was appropriately recommended
8,pct_cumulative,TRUE,FALSE,NA,Lower,Percent,0,1,Case Harm Rate,% of cases with at least one severely harmful error
9,normalized,TRUE,FALSE,NA,Lower,Absolute,0,50,Harmful Errors,Total number of severely harmful errors
10,nnh_cumulative,TRUE,FALSE,NA,Higher,Absolute,0,30,Number Needed to Harm,Expected number of cases before the model causes a severely harmful error
11,Runtime,TRUE,FALSE,NA,Lower,Absolute,0,250,Runtime,Inference time per case in seconds
6 changes: 2 additions & 4 deletions frontend/src/components/BarChartCard.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -1508,9 +1508,8 @@ export function BarChartCard({

return (
<section className="flex flex-col gap-5 rounded-3xl bg-[#f4f4f5] p-6 transition-all duration-[600ms] ease-[cubic-bezier(0.33,1,0.68,1)]">
<header className="flex flex-wrap items-start justify-between gap-4">
<header className="grid gap-4 md:grid-cols-[minmax(0,1fr)_auto] md:items-start">
<div className="flex flex-col gap-1">

<h2 className="text-xl font-semibold text-[#0c0d10]">
<button
type="button"
Expand All @@ -1534,9 +1533,8 @@ export function BarChartCard({
<p className="text-xs text-neutral-500">{metricDescription}</p>
) : null}
</div>
<div className="flex min-w-[12rem] flex-col items-end gap-2">
<div className="flex min-w-[12rem] flex-col items-end gap-2 md:justify-self-end">
<div className="flex w-full flex-col gap-1">

<select
id="bar-chart-metric-select"
value={metricId}
Expand Down
12 changes: 10 additions & 2 deletions frontend/src/components/NoharmInfoCard.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,16 @@ export function NoharmInfoCard({ className }: NoharmInfoCardProps) {
<div className="flex flex-col gap-1">
<dt className="text-xs font-semibold uppercase tracking-wide text-slate-500">Submissions</dt>
<dd>
An automated submission portal is in the works. In the meanwhile, please contact us if you
are interested benchmarking your model and inclusion in the leaderboard.
Please see the{" "}
<a
href="https://github.com/HealthRex/mast"
className="font-medium text-brand-600 hover:text-brand-700"
target="_blank"
rel="noopener noreferrer"
>
MAST GitHub Repository
</a>
{" "}for information and instructions on participating.
</dd>
</div>
<div className="flex flex-col gap-1">
Expand Down