Follow-up to #17: demos, an explicit API boundary, C++ support, and documentation

[LarryRuane]

If you're smoke-testing this, start here. It's still one header you copy in, so the fastest check is to drop the new protothread.h into something you already have and see whether it builds and behaves. Only three things can affect code you've already written:

• A protothread may no longer free the storage holding its own pt_thread_t. The scheduler marks a thread exited after its function returns, so freeing that storage from inside the thread is now a use-after-free, and sanitizers won't reliably report it. Free it from whoever owns it, after pt_join() or once pt_is_alive() is false.
• pt_set_atexit() is gone. if (pt_kill(&c->pt_thread)) { cleanup(c) ; } does the same thing, and it was only ever called in that one case anyway.
• Internal names picked up a pt_i_ prefix. Anything you were calling that didn't have one is unchanged and still public.

Everything else is additions (pt_join(), pt_is_alive(), timers, interrupt safety, PT_NO_MALLOC), documentation and CI. The commit count is high because nothing has been squashed, but there's no need to read through it.

---

Follow-up to #17, in nine groups of commits, each independent and reviewable on its own.

1. Check protothread_run() results, and document the context-struct idiom

The suite called protothread_run() 15 times and discarded the answer, including twice on a scheduler with nothing in it. Each call site now checks the result against the contract: false when nothing was ready, otherwise whether the ready list is still non-empty.

Verified they bite, by injecting bugs into protothread_run() and running both the old and new suites:

injected bug	old suite	new suite
always returns true	aborts	aborts
"nothing ready" path returns true	hangs (20s timeout)	caught at :34, names the condition
reports readiness before running	caught at :364, 1000-thread test	caught at :113, 2-thread test

The middle row is the point: that bug makes while (protothread_run(pt)) ; spin forever, so CI would burn its timeout rather than report a failure.

Also in this commit: the README's "you cannot use the function return value for your own purposes" — which reads as an apology — is replaced with a worked in/out parameter example, since the caller owns the callee's context and it survives blocking. A TODO entry records that *s = (struct protothread_s){0} must not replace protothread_init()'s explicit loop: measured freestanding at PT_NWAIT=1024, the loop needs no libc symbols at any optimization level, while the struct assignment makes gcc emit memset and clang memcpy + memset at -O0 — breaking the zero-libc guarantee in exactly the build used to debug on bare metal. And .gitignore picks up build*/ and .vscode/.

2. Collect the demos into demo/, and add two more

The thread-pool example was the only worked example, and it covered one of the three ways a protothread meets the outside world. protothread_pool_example.c becomes demo/pool.c, joined by:

• demo/async_io.c — no POSIX threads at all. Concurrent asynchronous I/O with one poll() loop turning completions into signals. ptaio -v traces every submission and completion.
• demo/helper_thread.c — one throwaway thread per blocking call, created and joined by the protothread that needs it.

The -v trace is why this commit is bigger than it looks. The straightforward version printed 64 submissions in index order followed by 64 completions in index order — a lock-step batch that teaches the reader the opposite of the truth about asynchronous I/O. Two changes fixed it: each protothread now runs a short sequence of I/Os, and the simulated device finishes after a pseudo-random delay. The pipes, the poll(), the blocking and the completion handler are all real; only the latency is simulated, and the comment says so. The 1 ms sleep in helper_thread.c makes the interleaving measurable — 64 helpers finish in 4 ms, not 64 ms, so the concurrency is demonstrated rather than asserted.

The demos also document offloading as the three arrangements it actually is — a thread per call, a standing pool, or a dedicated helper per protothread parked between calls (about 8 kB resident each against 64 bytes for the protothread) — rather than the earlier claim that a thread per protothread is always POSIX threads with extra steps. That is only true of giving each protothread a thread to run on.

3. Mark internal names with a pt_i_ prefix, and list the API

Header-only removed the boundary the linker used to draw: when there was a library, the non-static functions were the API. Now everything is static inline in a header and nothing distinguishes the two. 32 internal functions and macros now carry pt_i_ / PT_I_, protothread.h opens with the complete public API one line each, and the README carries the same list near the top.

A leading underscore — the obvious choice, and the one most header-only C libraries make — is not available. C reserves identifiers beginning with an underscore at file scope, and C++ additionally reserves any identifier containing a double underscore, so stb's stbi__ and sokol's _sg_ are both formally nonconforming. Six struct tags in this repo already had the reserved form (_pt_lock_t and friends) and now use the house _s suffix.

Three list primitives took a critical section that every caller already held (pt_i_link, pt_i_unlink, pt_i_find_and_unlink). PT_CRITICAL_* nests, so these were harmless — but on a target where entering one means touching PRIMASK they were not free. They now document that the caller must already be in a critical section.

That removal was verified rather than reasoned about: a harness defines PT_CRITICAL_ENTER/EXIT as a depth counter and asserts non-zero depth on entry to each function, and the suite runs clean under it across PT_DEBUG, PT_NWAIT and optimization settings, with assertions forced live under PT_DEBUG=0. The first version of that harness was vacuous — the quoted #include resolved to the real header rather than the instrumented copy — which a negative control caught by passing when it was required to fail.

4. Compile as C++, and keep it that way with CI

One line stood in the way: protothread_create() assigns malloc()'s void * to a typed pointer, which C++ will not do implicitly. A C-style cast is valid in both languages and costs C nothing.

CI now builds and runs a real protothread as C++ — yields, nesting through the semaphore, lock and timer helpers, and the computed goto — under g++ and clang++ at -std=c++11, c++17 and c++20. Compiling only the headers would have been cheaper and would have missed whether the machinery works at all.

The README claim is deliberately narrow: C code does not port unchanged, because every protothread function opens by recovering its context from env_t and each one needs a cast. Nor is any of this idiomatic C++ — C++20 coroutines are the native answer for new C++ code. What it offers is one scheduler shared between C and C++ translation units.

One finding worth flagging: clang++ rejects an initialized local declared after pt_resume() outright, because C++ forbids jumping into the scope of a variable with an initializer. That promotes the bug described under "Local variables" from a warning to a hard error — but only that one. The silent case, a variable initialized before pt_resume() and re-initialized on every resume, still compiles, so const remains the only thing that catches it.

5. README structure and wording (three small commits)

Drop the pointer to the multicore fork. The second line of the README sent readers to protothread-multicore, a fork of a much older version of this one that trades away determinism and the freedom from locking discipline — two of the three reasons to use protothreads. That repository is now archived with a note explaining what it was and why it stopped.

Move bare-metal use and interrupt safety near the top. "By default this library is not interrupt-safe" sat at line 494, after the example, the implementation walkthrough, the benchmarks and the versioning policy. Someone writing an ISR that calls pt_signal() reached it, if at all, after writing the bug — and the failure is silent, with no assertion to catch it. It now follows the API listing. The CMSIS PT_CRITICAL_* definitions and the interrupt-latency note were too detailed to lead with, so they became a subsection of Configuration next to where those macros are documented. No prose was rewritten and all internal links still resolve.

Say microcontroller. The README said "MCU" once, in an aside about code size, and otherwise relied on "bare metal" and "embedded" — accurate, but not the word someone deciding whether this runs on their STM32 searches for. It now appears in the two headline bullets, the opening line of the bare-metal section, and beside the toolchain list, which already named four microcontroller compilers without ever using the term.

6. Check the critical-section discipline in CI

Commit 3 removed three redundant critical sections on the grounds that every caller already held one. That property was then documented in comments and nothing more — and PT_CRITICAL_* are no-ops unless the application defines them, so a future change that started touching a list from thread context would break interrupt safety on a real target while passing every test in this repo.

PT_CRITICAL_ASSERT() makes the requirement checkable. It compiles to nothing by default (the freestanding build still needs zero libc symbols at every optimization level), and an application on a target where entering a critical section is observable can define it to audit its own discipline. CI defines it against a depth counter and runs the suite across PT_DEBUG, NDEBUG, PT_NWAIT and four optimization levels on both compilers.

Two things about the job itself:

• It runs its negative control first and fails the build if the control does not trap. The interesting failure mode here is not a library bug but a harness that quietly tests nothing — which is not hypothetical: the first version of this check, run locally, passed against an uninstrumented header because a quoted #include resolved next to the test file rather than to the instrumented copy. Defining a macro instead of rewriting the header removes that class of mistake entirely, so the job copies and edits no source at all.
• It uses __builtin_trap() rather than assert(), since NDEBUG and PT_DEBUG=0 are two of the configurations being checked and assert() would vanish in exactly those.

Verified end to end rather than by the control alone: with pt_i_add_ready()'s critical section removed, all 24 configurations fail; with it restored, all 24 pass.

7. Documentation: accuracy, reading order, and the case for the design (20 commits)

All README and TODO; no code changes.

Corrections — the docs were saying things that aren't true.

• The API reference grouped pt_signal(), pt_broadcast(), pt_kill() and pt_create() under "Either thread or non-thread execution context", which reads naturally as "from any OS thread or interrupt handler" — exactly where pt_signal() loses wakeups. Now split into Creating, waking and killing protothreads (call from the scheduler's OS thread; the rule is overlap, not identity) and Protothread system setup and teardown.
• The README taught nested protothread functions to take env_t and cast it. pt_call() is a direct call, so a nested function can take its real context type and be fully type-checked — the library's own semaphore, lock and timer code already did. Only top-level functions need env_t. The example was extracted from the README as written and compiled under gcc, clang, g++ and clang++; passing the wrong context to pt_call() is a hard error on both.
• Two C++ claims narrowed accordingly: only top-level functions need a cast.

Reading order. Bare-metal use and interrupt safety now follow the producer/consumer example, so a newcomer sees what a protothread is before meeting a lost-wakeup race diagram. Verified as a pure move: identical word count, every internal link resolves.

New arguments for the design.

• Wait channels: a condition variable without the pthread_cond_t; universal, since every blocking primitive here is ordinary protothread code over it (and Linux's futex is the same shape); and reducible to a busy-wait on a predicate with a delay inserted — with the one place that intuition misleads, which is lost wakeups.
• Stack sizing: protothreads don't abolish it, but reduce it from one worst-case guess per thread to one overall.
• Determinism made concrete: SAN/iQ's dbd_test, now called deterministic simulation testing, with FoundationDB's Flow as the independent industrial example. It says plainly that most users won't need this, and that making every external interface mockable is most of the work.

Presentation. API reference and Configuration entries now put each prototype inside its own description block. README code blocks use the conventional unspaced semicolon. C++20/C++11 escaped so markdown-it's ++ins++ extension no longer underlines half a paragraph. Removed a duplicate idle loop and a duplicated explanation of channels.

TODO. Three recorded design notes: a lock-free completion queue (and why it replaces only half of what a self-pipe does), a bounded-delay sweep with a lost-wakeup detector, and type-safe top-level protothreads with its measured cost — gcc refuses to inline any function containing a computed goto.

8. Remove pt_set_atexit(), finish the pt_i_ prefix, and add a changelog (6 commits)

pt_set_atexit() is removed. Whoever calls pt_kill() can do the cleanup, exactly equivalently, since the callback ran only when the kill found the thread, after the unlink, in the caller's context:

if (pt_kill(&c->pt_thread)) {
    cleanup(c) ;
}

It cost a pointer in every protothread, including in programs that never kill anything; was named after atexit() but never ran on normal exit; and could run a free() in interrupt context. Measured, not computed: the minimum RAM per protothread drops from 32 to 28 bytes on Cortex-M and from 64 to 56 on x86-64, taking the benchmark's memory ratio against pthreads from 256x to 293x.

The pt_i_ prefix now covers types and enum constants too, so "any name without the prefix is public API" is literally true: state_t, pt_critical_t, PT_RETURN_*, pt_return_e/_s, pt_lock_state_t, the PT_LOCK_* request states and pt_time_diff_t. state_t is deleted rather than renamed, since it was a second typedef for the type protothread_t already names, and repeating that typedef is an error under -std=c99.

CHANGELOG.md, in Keep a Changelog format. Building it from the history exposed that the README's "Upgrading from 1.x" notes omitted the renamed internals and struct tags; they are listed now.

README:

• The API section shows only the core — 18 entries in protothread.h, of which the producer/consumer example uses 9. Semaphores, reader-writer locks and timers are presented as optional layers built on it, with their reference under a new "Built on top" heading. The headers, tests and CI coverage are unchanged.
• A paragraph on why the blocking macros are lowercase: readability, with precedent in the C standard's own setjmp, assert and va_arg and the kernel's wait_event().

Verified before pushing: the 70-configuration matrix, all twelve language standards including C99, ASan/UBSan, zero libc symbols freestanding, C++11 and C++20, the demos, and the critical-section and C++ CI jobs run verbatim from the workflow.

9. Nothing may depend on pt_signal() waking only one waiter (4 commits)

A wakeup is a hint that a condition may have changed, never a promise, so waking one waiter rather than all of them should be an optimization that correct code never relies on. PT_SIGNAL_WAKES_ALL=1 makes pt_signal() behave as pt_broadcast(); it defaults to 0 and compiles to nothing. A new CI job, signal wakes all, builds the test suite and the three demos that way on gcc and clang.

The job checks its own switch first. With two consumers and a producer on one channel, the program stalls when pt_signal() wakes one waiter — a consumer absorbs the wakeup the producer needed — and completes when it wakes everyone. A macro that silently stopped working would otherwise leave the job green while testing nothing.

That same program is why the README's new text is careful about the converse. Waking everyone is always safe, but pt_signal() is a safe substitute for pt_broadcast() only when every waiter on the channel could use the event. The Wait channels section now says so, and explains where the producer/consumer example uses while rather than if. The library's own semaphores, locks and timers never call pt_signal() at all.

Also: README "What you get" tightened, the CMake find module described in plain terms, and a typo.

pt_broadcast() is now the README's stated default, as the Unix kernel's wakeup() always was. pt_signal() is safe only when every waiter on the channel waits for the same condition, the woken waiter is certain to run, and it leaves the condition false or passes the wakeup on — properties of the whole program, not of the call. What pt_signal() saves is a thundering herd, which needs many waiters on one channel, and on a microcontroller that rarely exists; where it does, splitting the channel is free. The producer/consumer example and its expansion now use pt_broadcast() to match; with one producer and one consumer the behavior is identical.

---

Verified locally, beyond what CI runs: 70-configuration C matrix on gcc and clang (PT_DEBUG, NDEBUG, PT_NWAIT of 1/4/1024, -O0 through -Os), twelve language standards, six C++ configurations, zero libc symbols under -ffreestanding at every optimization level, ASan, UBSan, TSan, the three demos, and the cmake build and install — all under -Wall -Wextra -Werror.

🤖 Generated with Claude Code

[LarryRuane]

Hi @TragicWarrior, just wondering if you might want to review (and merge) this, no pressure if you're too busy, hope it's not too much. It's turned into a general improvement/cleanup PR. (Claude code has been amazingly helpful.) If you're not available, I can just self-merge but wanted to give you a chance first. Thanks!

There's really only two changes that are non-fluff (could definitely benefit from review):

• protothread.h, commit 3 — the three removed critical sections are the only change in the PR that could cause a real bug on an interrupt-driven target. I verified it with a depth-counter harness (see commit comment). Actually, just added an 8th commit to test this in CI.
• The API list, same commit — whether the public/internal seems right to you.

[LarryRuane]

Hi @TragicWarrior, I'm about to make more changes to this PR, so please don't merge. Actually, I'll set it to DRAFT for now.

I have a question about pt_set_atexit(), which you added back in 2019 (43224c6).

While reorganizing the API reference I've started wondering whether we still need it, and since v2.0.0 isn't tagged yet, now is a good time to decide. The case against:

• Whoever calls pt_kill() can usually do the cleanup itself, right after the call.
• It costs a pointer in every pt_thread_t, even in programs that never kill anything. Removing it would drop the minimum RAM per protothread from 32 to 28 bytes on a 32-bit target.
• Despite the name, it runs only when a protothread is killed, never when it returns normally, unlike C's atexit().
• If pt_kill() is called from an interrupt handler (allowed once PT_CRITICAL_* are defined), the callback runs in interrupt context, where typical cleanup like free() isn't safe.

The strongest case for keeping it is that generic clean-up code — a supervisor, say — that holds only a pt_thread_t * and doesn't know the context's type, so it can't do type-specific cleanup itself. That's legitimate, though application code could also handle it with its own registry of protothreads and cleanup functions.

Do you still use it, or know of code that does? Nothing in this repo uses it apart from its own test. If you don't have a strong opinion (or don't mind changing your existing code that uses it), I'd prefer to nuke it for version 2.0.0. Thanks!

[LarryRuane]

One more thing, so I'm not contradicting myself: when I closed #15 I wrote that pt_set_atexit() "turned out to be the natural hook for running a C++ destructor when a protothread is killed", and TODO.md says the same.

Thinking about it again, that's weaker than it sounded. A C++ wrapper would be templated on the context type, so the code that kills a protothread already knows the type and can run the destructor itself — call pt_kill(), then destroy the context — without a function pointer stored in every protothread. The same trick works in plain C: a macro can generate a typed pt_kill_foo(foo_ctx_t *) that kills and then runs the cleanup, fully type-checked.

If you are using it, the change on your side should be small. The callback only ran when the kill actually found the thread, so at each call site this does exactly what it did:

if (pt_kill(&c->pt_thread)) {
    cleanup(c) ;
}

[LarryRuane]

Pushed a few more updates...

It's ready for your smoke test, we can un-draft this once you've had a chance to rebuild.

2.0.0 has some deliberate breaking changes. Here's what I'd expect you to hit, most likely first:

1. No library to link. It's header-only now, so the protothread-static and protothread-shared CMake targets are gone, and pkg-config no longer emits -lprotothread. If you use add_subdirectory and link one of those targets, link the protothread target instead, or nothing at all. find_package(PROTOTHREAD) still works; ${PROTOTHREAD_LIBRARIES} is just empty.
2. pt_set_atexit() is removed. Where you registered a callback, do the cleanup at the call site instead. This runs in exactly the circumstances the callback did:

if (pt_kill(&c->pt_thread)) {
    cleanup(c) ;
}

3. Standard headers no longer come through protothread.h: <string.h> never, <assert.h> only while PT_DEBUG is on, and <stdlib.h> only without PT_NO_MALLOC. Code using memset, assert or malloc without its own #include will break — and code relying on assert can build fine in debug and fail only with PT_DEBUG=0. If you can, please rebuild with -DPT_DEBUG=0 as well as your usual configuration.
4. Internal names now have a pt_i_ prefix, so anything that reached into internals won't compile: for example pt_get_protothread() (use pt_get_pt(c)), state_t (use protothread_t, the same type), or struct _pt_lock_t (use the pt_lock_t typedef).

One behavioral change rather than a build error: reader-writer locks now grant requests in arrival order.

The full list is in CHANGELOG.md and the README's "Upgrading from 1.x" section. Anything that breaks and isn't on this list is exactly what I'm hoping the smoke test turns up, so please let me know either way.