advanced_usage.md

Documentation: Advanced Usage (Multi-threaded & Cancellation)

This section explains the advanced implementation for bulk image processing using Parallel.ForEach and CancellationToken.

1. Thread-Safe Progress Tracking (ProcessResult)

When processing images across multiple threads, standard primitive variables (like int) can suffer from race conditions.

• ProcessResult Class: Acts as a data transfer object (DTO) to safely collect metrics (Total, Processed, Success, Fail) from concurrent threads.
• Interlocked.Increment: Used inside the loop to safely increment counters without the high overhead of a standard lock block.
• ConcurrentBag: A thread-safe collection used to gather the resulting file paths seamlessly from multiple threads simultaneously.

2. Pre-Filtering & Optimization (IsWebPLossless)

To prevent redundant encoding tasks and save CPU cycles, the code pre-filters the target files before handing them over to the native encoder.

• The Challenge: If you accidentally re-encode an already optimized WebP image with a lossy configuration, it causes unnecessary quality degradation and CPU waste.
• The Solution (IsWebPLossless): It opens the file with FileShare.ReadWrite to avoid file-locking conflicts with other processes. It then reads only the first 64 bytes of the file header to look for the "VP8L" signature (which identifies a WebP Lossless image). This lightweight I/O check allows the processor to quickly skip or accept files without decoding the entire image into memory.

3. Isolated Memory Management & Image Decoding (LoadAsRgba)

Managed image libraries sometimes hold onto file handles longer than expected, which can block other operations in high-throughput parallel setups.

• SixLabors.ImageSharp Dependency: This implementation relies on the SixLabors.ImageSharp NuGet package for cross-platform image decoding. It explicitly parses the image into the Rgba32 format, ensuring the byte-order strictly matches what the native WebpEncoder.EncodeRGBAtoWebP expects.
• Immediate I/O Release: LoadAsRgba utilizes File.ReadAllBytes to instantly dump the image into memory and release the OS file handle immediately.
• Decoupled Decoding: The image parsing happens entirely inside an isolated MemoryStream. Once the raw pixel byte array (rgba) is extracted, the managed image objects are disposed of immediately, keeping the managed memory footprint stable during heavy parallel runs.

4. Granular Cancellation Response

Cancellation in multi-threaded environments can be sluggish if not handled carefully. This implementation ensures immediate responsiveness by scattering token.IsCancellationRequested checks at critical lifecycle stages:

1. **Before entering ProcessFile**: Drops out of the Parallel.ForEach loop instantly via state.Stop().
2. Before heavy image decoding: Prevents wasting CPU on parsing new images if a cancel request was just made.
3. Before native encoding: Ensures we do not invoke the native WebpEncoder.EncodeRGBAtoWebP if the user has already clicked "Cancel".

---

using System.Collections.Concurrent;
using System.Runtime.InteropServices;

namespace WebPConv
{
    // Holds the execution metrics and successfully generated temporary file pairs.
    public class ProcessResult
    {
        public int Total;
        public int Processed;
        public int Success;
        public int Fail;

        public ConcurrentBag<(string original, string temp)> Results = [];
    }

    public class WebpBatchProcessor
    {
        // Minimum valid byte size for an output WebP file to pass verification.
        private const long MinValidWebpSizeBytes = 100;

        public async Task<ProcessResult> RunAsync(
            string[] items,
            bool losslessMode,
            IProgress<int>? progress = null,
            CancellationToken token = default)
        {
            var result = new ProcessResult();

            var targets = items.SelectMany(item =>
                Directory.Exists(item)
                ? Directory.GetFiles(item, "*.*", SearchOption.AllDirectories)
                : [item]
            ).Where(IsTargetImage).ToArray();

            result.Total = targets.Length;

            if (result.Total == 0)
                return result;

            await Task.Run(() =>
            {
                // Throttling parallel degrees to prevent CPU starvation
                int maxParallel = Math.Min(Environment.ProcessorCount / 2, 4);

                var options = new ParallelOptions
                {
                    MaxDegreeOfParallelism = Math.Max(1, maxParallel)
                };

                // Multi-threaded batch processing
                Parallel.ForEach(targets, options, (file, state) =>
                {
                    // Check cancellation before processing the next file
                    if (token.IsCancellationRequested)
                    {
                        state.Stop();
                        return;
                    }

                    try
                    {
                        ProcessFile(file, losslessMode, result, token);
                    }
                    catch
                    {
                        Interlocked.Increment(ref result.Fail);
                    }
                    finally
                    {
                        int currentProcessed = Interlocked.Increment(ref result.Processed);

                        progress?.Report(currentProcessed);
                    }
                });
            }, token);

            return result;
        }

        private static void ProcessFile(string file, bool losslessMode, ProcessResult result, CancellationToken token)
        {
            if (token.IsCancellationRequested) return;

            string ext = Path.GetExtension(file).ToLower();
            bool shouldProcess = false;

            if (ext == ".png" || ext == ".bmp")
            {
                shouldProcess = true;
            }
            else if (ext == ".webp")
            {
                if (!losslessMode && IsWebPLossless(file))
                {
                    shouldProcess = true;
                }
            }

            if (!shouldProcess) return;

            string webp = Path.ChangeExtension(file, ".webp");
            string tempPath = webp + ".tmp";

            if (File.Exists(tempPath)) return;
            if (ext != ".webp" && File.Exists(webp)) return;
            
            // Check cancellation before heavy image decoding
            if (token.IsCancellationRequested) return;

            byte[] rgba = LoadAsRgba(file, out int width, out int height);

            // Check cancellation before native encoding
            if (token.IsCancellationRequested) return;

            int ok = WebpEncoder.EncodeRGBAtoWebP(
                rgba,
                width,
                height,
                width * 4,
                90f,
                losslessMode ? 1 : 0,
                out nint ptr,
                out nuint size
            );

            // Handles encoder failures securely.
            if (ok == 0)
            {
                Interlocked.Increment(ref result.Fail);
                return;
            }

            ulong s = size.ToUInt64();
            // Validates memory bounds before copying data.
            if (s == 0 || s > int.MaxValue)
            {
                WebpEncoder.FreeWebP(ptr);
                Interlocked.Increment(ref result.Fail);
                return;
            }

            byte[] data = new byte[(int)s];
            Marshal.Copy(ptr, data, 0, (int)s);

            // CRITICAL: Always free the native memory allocation
            WebpEncoder.FreeWebP(ptr);

            File.WriteAllBytes(tempPath, data);

            var info = new FileInfo(tempPath);

            // Verifies output integrity using the size threshold.
            if (info.Exists && info.Length > MinValidWebpSizeBytes)
            {
                result.Results.Add((file, tempPath));
                Interlocked.Increment(ref result.Success);
            }
            else
            {
                Interlocked.Increment(ref result.Fail);

                if (File.Exists(tempPath))
                    File.Delete(tempPath);
            }
        }

        // Checks if the extension matches supported source image formats.
        private bool IsTargetImage(string path)
        {
            string ext = Path.GetExtension(path).ToLower();
            return ext == ".png" || ext == ".bmp" || ext == ".webp";
        }

        // Inspects file headers to detect the WebP lossless signature (VP8L).
        private static bool IsWebPLossless(string path)
        {
            using var fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);

            byte[] buffer = new byte[64];
            int read = fs.Read(buffer, 0, buffer.Length);

            if (read < 16) return false;

            // Scans the header buffer for the VP8L magic bytes.
            for (int i = 0; i < read - 3; i++)
            {
                if (buffer[i] == (byte)'V' &&
                    buffer[i + 1] == (byte)'P' &&
                    buffer[i + 2] == (byte)'8' &&
                    buffer[i + 3] == (byte)'L')
                {
                    return true;
                }
            }

            return false;
        }

        // Loads image pixels to an unmanaged-compatible RGBA byte array.
        private static byte[] LoadAsRgba(string path, out int width, out int height)
        {
            // Read all bytes instantly to close the file handle immediately
            byte[] fileBytes = File.ReadAllBytes(path);
            using var ms = new MemoryStream(fileBytes);

            using var image = SixLabors.ImageSharp.Image.Load<SixLabors.ImageSharp.PixelFormats.Rgba32>(ms);

            width = image.Width;
            height = image.Height;

            byte[] rgba = new byte[width * height * 4];
            image.CopyPixelDataTo(rgba);

            return rgba;
        }
    }
}