`rustix::termios::tcgetpgrp` is unsound on macOS

[taylordotfish]

The libc backend for rustix::termios::tcgetpgrp passes the result of libc's tcgetpgrp() directly to Pid::from_raw_unchecked on non-Linux platforms; only Linux has a check to make sure the PID isn't 0. However, tcgetpgrp() can return 0 on non-Linux platforms too, notably macOS.

Specifically, if there is no process running on the terminal (e.g., immediately after creating a PTY), tcgetpgrp() will return 0. On macOS, this means 0 will get passed to Pid::from_raw_unchecked, which is UB.

Reproduction

Must be run on macOS. Might fail on other non-Linux platforms but I haven't tested.

cargo new rustix-issue
cd rustix-issue

Put the following in Cargo.toml:

[package]
name = "rustix-issue"
version = "0.1.0"
edition = "2024"

[dependencies.rustix]
version = "1.1.4"
features = ["pty", "termios"]

Put the following in src/main.rs:

use rustix::pty::{OpenptFlags, openpt};
use rustix::termios::tcgetpgrp;

fn main() {
    let fd = openpt(OpenptFlags::RDWR | OpenptFlags::NOCTTY).expect("openpt");
    match tcgetpgrp(&fd) {
        Ok(pid) => println!("tcgetpgrp() -> {pid}"),
        Err(e) => println!("tcgetpgrp() -> {e}"),
    }
}

Then run cargo run. As I don't use macOS I asked @ellie to run this; here's the output:

thread 'main' (14903657) panicked at /Users/ellie/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rustix-1.1.4/src/pid.rs:61:9:
assertion failed: raw > 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

We hit the debug assertion in Pid::from_raw_unchecked, which indicates UB.