Concurrent multi-processors never flush at interpreter exit

[dwin-gharibi]

Describe your environment

Labels: bug, sdk, trace, logs, data-loss
Affected packages: opentelemetry-sdk
Found on: main @ 0a5d76b6
Environment: CPython 3.12

What happened?

TracerProvider and LoggerProvider register their shutdown with atexit.register(). concurrent.futures registers its cleanup through threading._register_atexit(), and CPython runs threading._shutdown() before the atexit queue.

So by the time the provider's shutdown runs, the thread pool is already closed. _submit_and_await raises RuntimeError: cannot schedule new futures after shutdown, and the underlying BatchSpanProcessor / BatchLogRecordProcessor is never shut down - its buffered telemetry is simply lost.

Steps to Reproduce

from opentelemetry.sdk.trace import TracerProvider, ConcurrentMultiSpanProcessor
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor, SpanExporter, SpanExportResult,
)

class Exporter(SpanExporter):
    def export(self, spans):
        for span in spans:
            print("EXPORTED", span.name)
        return SpanExportResult.SUCCESS
    def shutdown(self):
        print("EXPORTER_SHUTDOWN")

provider = TracerProvider(active_span_processor=ConcurrentMultiSpanProcessor(2))
provider.add_span_processor(BatchSpanProcessor(Exporter(), schedule_delay_millis=600000))
with provider.get_tracer(__name__).start_as_current_span("span-flushed-at-exit"):
    pass
# interpreter exits here; nothing is printed

Expected Result

The buffered span is exported and the exporter's shutdown is called, exactly as happens with the default SynchronousMultiSpanProcessor.

Actual Result

Exception ignored in atexit callback: TracerProvider.shutdown
Traceback (most recent call last):
  File ".../sdk/trace/__init__.py", line 293, in shutdown
    self._submit_and_await(lambda sp: sp.shutdown)
  File ".../sdk/trace/__init__.py", line 272, in _submit_and_await
    future = self._executor.submit(func(sp), *args, **kwargs)
RuntimeError: cannot schedule new futures after shutdown

# stdout is empty - the span was never exported

# CPython's finalization order, observed directly
ORDER: threading._shutdown hooks -> atexit hooks

# control: an explicit mid-program shutdown works fine
['EXPORTED 1', 'EXPORTER_SHUTDOWN', 'MANUAL_SHUTDOWN_OK']

Additional context

Every buffered span and log record is lost on every clean exit, for anyone using the concurrent multi-processors. Both signals are affected.

It is quiet in the worst way: the traceback is printed as an ignored atexit error, so the process still exits 0 and CI stays green. The telemetry most likely to be lost is the telemetry from the end of the run - often exactly the part you care about when diagnosing a shutdown.

force_flush has the same failure mode when called late.

Would you like to implement a fix?

Yes

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}