Allow multi-thread streaming using the same weight loaded once in memory

[kyteinsky]

I'm trying out this project for a live transcription usecase in a call. We use vosk at the moment (in https://github.com/nextcloud/live_transcription/) which supports loading the model weights once in the memory and then starting new threads with light-er recognize objects reading the same weights to process the output but keeping their own state and cache.
This is not possible at this moment due to a mutex

parakeet.cpp/src/ggml_graph.cpp

Lines 28 to 35 in e75de9b

	// Serializes access to the process-global Backend. The old run_graph allocated
	// a fresh per-call ggml context (so concurrent transcribes were independent);
	// the shared Backend (one gallocr + pending-input list, not re-entrant) is not,
	// so we serialize compute() across threads. In practice inference is driven
	// from a single thread per process (the parallelism is inside the graph's
	// worker threads), so this lock is uncontended; it only guards against a caller
	// that drives transcribe() concurrently from multiple threads.
	std::mutex g_backend_mutex;

One process can process only one stream at a time.

I'm not familiar with the code so did an experiment and asked AI if there is a possibility to work like llama.cpp here which uses slots to entertain parallel requests using the same loaded weights, and works with the same underlying ggml library.
It has successfully changed the code to make it possible for multiple threads to transcribe at the same time, in the same process by using a new Backend in each thread as opposed to one shared Backend + mutex guard.
The tests ran on an AMD CPU but theoritically should not cause issues with GPU systems.

Below are some reproduction steps and the patch:

parakeet-thread-local-backend.patch
concurrent_streams.py

python concurrent_streams.py --lib ./libparakeet.so --model ./nemotron-3.5-asr-streaming-0.6b-q8_0.gguf --threads 4 ./audio/en_9min_16k* --seconds 90