[Bug]: Intermittent `ValueError: The passed figure is not managed by pyplot` with ipympl (`%matplotlib widget`)

[michitaro]

Bug summary

When using Matplotlib with ipympl (%matplotlib widget) in JupyterLab, I intermittently get:

ValueError: The passed figure is not managed by pyplot

This happens with standard pyplot usage (plt.figure, plt.subplots(num=fig), plt.sca(ax)), without explicitly constructing canvases.

Code for reproduction

%matplotlib widget
import time
import matplotlib.pyplot as plt

def try_once():
    fig_num = 1
    plt.close(fig_num)
    fig = plt.figure(fig_num)

    fig, axs = plt.subplots(10, 10, num=fig, squeeze=False)

    for ax in axs.flat:
        plt.sca(ax)  # <-- intermittently raises
        plt.plot([1, 2, 3])
        fig.canvas.draw_idle()
        time.sleep(0.001)

for i in range(300):
    try:
        try_once()
    except Exception as e:
        print('FAILED at iteration', i, '->', repr(e))
        raise

Actual outcome

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 20
     18 for i in range(300):
     19     try:
---> 20         try_once()
     21     except Exception as e:
     22         print('FAILED at iteration', i, '->', repr(e))

Cell In[2], line 13, in try_once()
     10 fig, axs = plt.subplots(10, 10, num=fig, squeeze=False)
     12 for ax in axs.flat:
---> 13     plt.sca(ax)  # <-- intermittently raises
     14     plt.plot([1, 2, 3])
     15     fig.canvas.draw_idle()

File ~/ipympl-debug/issue/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py:1376, in sca(ax)
   1372 # Mypy sees ax.figure as potentially None,
   1373 # but if you are calling this, it won't be None
   1374 # Additionally the slight difference between `Figure` and `FigureBase` mypy catches
   1375 fig = ax.get_figure(root=False)
-> 1376 figure(fig)  # type: ignore[arg-type]
   1377 fig.sca(ax)

File ~/ipympl-debug/issue/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py:995, in figure(num, figsize, dpi, facecolor, edgecolor, frameon, FigureClass, clear, **kwargs)
    993 root_fig = num.get_figure(root=True)
    994 if root_fig.canvas.manager is None:
--> 995     raise ValueError("The passed figure is not managed by pyplot")
    996 elif (any(param is not None for param in [figsize, dpi, facecolor, edgecolor])
    997       or not frameon or kwargs) and root_fig.canvas.manager.num in allnums:
    998     _api.warn_external(
    999         "Ignoring specified arguments in this call because figure "
   1000         f"with num: {root_fig.canvas.manager.num} already exists")

ValueError: The passed figure is not managed by pyplot

Expected outcome

Mosaicked plots will be shown without errors.

Additional information

Observed state at failure

At the moment the exception occurs:

• root_fig.canvas.manager is None
• but the corresponding FigureManager still exists in matplotlib._pylab_helpers.Gcf (so pyplot still manages the figure)

So pyplot’s “managed figure” check is making a false negative.

Where the exception comes from

plt.sca(ax) calls pyplot.figure(fig) internally.
When passed a Figure, pyplot.figure() currently checks only:

root_fig = num.get_figure(root=True)
if root_fig.canvas.manager is None:
    raise ValueError("The passed figure is not managed by pyplot")

In this bug, canvas.manager is temporarily/incorrectly None even though the manager still exists in Gcf.

Proposed fix

If root_fig.canvas.manager is None, fall back to a Gcf lookup to find the manager whose m.canvas.figure is root_fig, and reattach it.

manager = root_fig.canvas.manager
if manager is None:
    for m in _pylab_helpers.Gcf.get_all_fig_managers():
        if getattr(getattr(m, "canvas", None), "figure", None) is root_fig:
            manager = m
            root_fig.canvas.manager = m
            break
if manager is None:
    raise ValueError(...)

This same logic eliminates the failures in my environment (0/60) when applied as a small monkey patch around pyplot.figure().

This resembles matplotlib/matplotlib#19380 (same structural cause: FigureCanvasBase.__init__ sets self.manager=None after replacing figure.canvas).

Operating system

Ubuntu 24

Matplotlib Version

3.10.8

ipympl Version

0.10.0

Matplotlib Backend

widget

Python version

Python 3.12.3

Jupyter version

jupyter lab --version: 4.5.4

Installation

uv

[Chirag3841]

@michitaro Use the object-oriented API instead of plt.sca(ax) — replace:
plt.sca(ax)
plt.plot(...)
with ax.plot(...)
also avoid mixing plt.figure() with plt.subplots(num=fig) — just use plt.subplots(num=fig_num).
This prevents the pyplot manager desynchronization causing the error.

[michitaro]

Thank you for your reply. Your suggestion did indeed solve the issue.

I think a warning such as 'avoid using pyplot.sca in a multithreaded environment' will be helpful when the user does so.

[timhoffm]

While using the object-oriented API is indeed the recommended solution here, in theory plt.sca() should be allowed.

Does the error still happen if you leave out fig = plt.figure(fig_num). Passing an existing figure to plt.subplots() is not intended behavior and it's likely that this is causing the inconsistent figure state. If that is the case, the follow-up action here would be to raise if num is passed to plt.subplots() and a figure with that number already exists.

[Aaratrika-Shelly]

Hello, this is a really interesting discussion. I decided to follow up on @timhoffm suggestion to see if the error disappears when you don't mix plt.figure() and plt.subplots().
I can confirm that when I remove the initial plt.figure(fig_num) call and only use plt.subplots(num=fig_num), the ValueError from plt.sca() no longer occurs in the user's reproduction code.
This seems to confirm your hypothesis that the mixed calls are causing the inconsistent state.
Based on that, I would be happy to work on the follow-up action you proposed: adding a check to plt.subplots() to raise a ValueError if the num parameter corresponds to a figure that already exists.
Would that be fine?

[timhoffm]

@Aaratrika-Shelly yes, you are welcome to create a pull request.

To immediately set you on the right track: Using fignum_exists() here will do the trick.

Please add a test and an API change note. Note: We don't need a deprecation because the current behavior is unsafe and directly prohibiting it is the best we can do.

[michitaro]

BTW, the version of ipykernel was critical.
The following are the versions of the related libraries and ipykernel, along with the error counts of the tests.

JupyterLab 4.5.4 / IPython 9.10.0 / matplotlib 3.10.8 / ipympl 0.10.0 / Python 3.12.3

   ipykernel 6.27.1: 0/30  
   ipykernel 6.29.5: 0/30  
   ipykernel 7.0.0: 13/30  
   ipykernel 7.2.0: 8/30

[timhoffm]

Skimming ipykernel 7.0 changelog, there's nothing obvious. But there's some weird interplay at work anyway, possibly, timing and events. Anyway, I believe not allowing to reuse an existing figure in plt.subplots() is the right fix.

[rcomer]

I can confirm that when I remove the initial plt.figure(fig_num) call and only use plt.subplots(num=fig_num), the ValueError from plt.sca() no longer occurs in the user's reproduction code.

I still get the error with the stated change. Do you know what we are doing differently?

[rcomer]

I note that, since matplotlib/matplotlib#29855, the error does not exist on main as you now can pass an unregistered figure to plt.figure.

[timhoffm]

Yeah, we also don't want to register a figure as a side-effect of passing it to subplots(). That's an independent reason to do matplotlib/matplotlib#31230, even if it is not the root cause of the bug.

[ianthomas23]

The ipykernel relevance is that >=7 supports subshells (separate threads) which will be used by default for the ipywidgets communications, making some things both faster and multithreaded. There is some protection in ipympl for this, but I suspect more is needed regardless of the changes discussed here.

[rcomer]

Should this issue be moved to the ipympl repo?

[ianthomas23]

Should this issue be moved to the ipympl repo?

That is probably sensible.