Serialize only the values the JSON encoder cannot take

[T4rk1n]

Fixes #4065.

With orjson installed, plotly's to_json_plotly dumps the whole payload, and if one value is not orjson-native (a component, a go.Figure, a pd.Timestamp) it walks every value in Python and dumps again. Layouts always take that path. Without orjson, everything goes through the stdlib walk.

dash._utils.to_json now goes through a new dash/_json.py:

• With orjson: orjson.dumps with a default= hook, so only the values orjson can't take are converted.
• Without orjson (or with plotly's engine set to "json"): msgspec, which Dash already depends on through shared storage, with the same kind of hook.
• Anything either encoder fails on goes to to_json_plotly, so errors and callback validation messages stay the same.

Numbers

From the serialization benchmark in #4055, 10 traces x 20k points, median ms:

case	orjson before	orjson after	no orjson before	no orjson after
list of float + Div	40	6	52	9
list of datetime + Div	171	14	350	13
list of datetime + Timestamp	172	14	351	13
pd.DatetimeIndex + Div	50	20	76	47
list of datetime as go.Figure / layout	375	234	566	232

Payloads that were already orjson-native take the same time as before (the hook is never called).

The go.Figure / layout rows are still bound by to_dict()'s deepcopy in plotly. That needs a plotly.py change (a _to_json_dict that skips the copy) and will be a follow-up.

Output differences

• With orjson: byte-identical to before on the values covered by the tests, except numpy datetime64 inside a figure, which now always uses orjson's short form (2024-01-01T00:00:00), as plotly's fast path already did. A datetime pd.Series with NaT now serializes (plotly raised).
• Without orjson (msgspec):
  • A tz-aware UTC datetime gives ...Z instead of +00:00.
  • Sets, bytes (base64), timedelta and np.timedelta64 serialize instead of raising.
  • NaN/bool dict keys format differently, np.datetime64 scalars give an ISO string, non-ASCII is emitted raw.
  • msgspec writes non-finite Decimals and Decimal dict keys as invalid JSON, so the output's syntax is checked (decoding to msgspec.Raw, about 1 ms per 8 MB) and anything invalid goes to plotly. Non-finite Decimals and np.datetime64('NaT') give null, and circular values raise ValueError, as in plotly's json engine.

Tests

• tests/unit/test_json.py: parity with plotly on both engines, a test that counts values visited by plotly's slow walk (fails before the fix), NaT handling, and unserializable values still raising TypeError, and non-finite / circular values matching plotly.
• tests/integration/callbacks/test_serialization.py: an app returning a figure, a component and a pd.Timestamp in one response, on both engines.

plotly/plotly.py#5813 removes the go.Figure deepcopy and adds a msgspec engine to plotly itself. Once it ships, a follow-up here can call _to_json_dict() instead of to_plotly_json() for figures, so the figure and layout rows above drop to the figure-alone time.

[github-actions]

Dash performance benchmarks

⚠️ regressions to review

	scenario	metric	p90 (ms)	median	growth	baseline p90	note
⚠️	full_children_replace	replace_ms	4421.6	1597.4	21.62x	4697.1	1.5x baseline (norm)
✅	callback_chain	chain_ms	339.7	328.7	0.91x	499.1
✅	callback_chain	graph_ms	1.3	1.3	1.0x	2.4
✅	callback_fanout	fanout_ms	58.7	50.0	0.93x	92.5
✅	deep_nesting	render_ms	37.9	35.7	1.02x	56.8
✅	initial_render_large	render_ms	319.0	314.7	0.98x	694.4
✅	initial_render_small	render_ms	63.2	58.9	1.07x	104.0
✅	patch_append_nested	append_ms	100.8	66.8	2.47x	192.3
✅	patch_append_toplevel	append_ms	77.3	62.8	2.28x	140.2
✅	patch_scalar_update_large	update_ms	98.1	83.3	0.95x	202.9
✅	wildcard_all_resolve	wildcard_ms	187.6	180.6	0.94x	313.6
✅	wildcard_all_resolve	graph_ms	0.7	0.7	1.0x	1.3

growth = late-third / early-third per-op time; ~1 is flat, a large value means the per-op cost scales with accumulated state.

machine scale vs baseline: 0.61x - divided out of the baseline ratios so they compare like for like (the absolute warn/fail ceilings are left un-scaled); calibrated on initial_render_small.

[Aaron-Wrote-This]

Ah yes I noticed this, having orjson installed was sometimes slower than without it! Glad to see it being addressed!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud