diff --git a/docs/source/conf.py b/docs/source/conf.py index 1a2ef3e5e5f..eaeb8ede9b5 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -230,7 +230,7 @@ ], "show_version_warning_banner": True, "use_edit_page_button": True, - "header_links_before_dropdown": 8, + "header_links_before_dropdown": 9, "navbar_align": "left", "navbar_start": ["navbar-logo", "version-switcher"], "navbar_center": ["navbar-nav"], diff --git a/docs/source/desktop-backends.md b/docs/source/desktop-backends.md deleted file mode 100644 index c894e01843c..00000000000 --- a/docs/source/desktop-backends.md +++ /dev/null @@ -1,27 +0,0 @@ -(desktop-backends)= -# Backends - -Available hardware acceleration backends for desktop platforms. - -## Linux Backends - -- {doc}`desktop-xnnpack` — XNNPACK (CPU acceleration) -- {doc}`desktop-openvino` — OpenVINO (Intel hardware optimization) - -## macOS Backends - -- {doc}`backends/coreml/coreml-overview` — CoreML (recommended for Apple Silicon) -- {doc}`backends/mps/mps-overview` — Metal Performance Shaders (Apple Silicon GPU) -- {doc}`desktop-xnnpack` — XNNPACK (CPU acceleration) - -## Windows Backends - -- {doc}`desktop-xnnpack` — XNNPACK (CPU acceleration) -- {doc}`desktop-openvino` — OpenVINO (Intel hardware optimization) - -```{toctree} -:hidden: -desktop-xnnpack -desktop-openvino -backends/coreml/coreml-overview -backends/mps/mps-overview diff --git a/docs/source/desktop-coreml.md b/docs/source/desktop-coreml.md deleted file mode 100644 index 48271326d87..00000000000 --- a/docs/source/desktop-coreml.md +++ /dev/null @@ -1 +0,0 @@ -```{include} backends-coreml.md diff --git a/docs/source/desktop-linux.md b/docs/source/desktop-linux.md new file mode 100644 index 00000000000..25808777ea3 --- /dev/null +++ b/docs/source/desktop-linux.md @@ -0,0 +1,107 @@ +# Linux Desktop Deployment + +ExecuTorch provides comprehensive support for Linux environments, enabling high-performance model execution across a wide range of hardware configurations. The runtime leverages optimized backends like XNNPACK for CPU execution and OpenVINO for Intel hardware acceleration. + +This guide details the system requirements, available backends, and steps to build and run ExecuTorch natively on Linux distributions. + +## Prerequisites + +ExecuTorch is actively tested and supported on several major Linux distributions. Ensure your environment meets the following minimum requirements: + +- **Operating System**: CentOS 8+, Ubuntu 20.04.6 LTS+, or RHEL 8+ [1]. +- **Compiler Toolchain**: `g++` version 7 or higher, `clang++` version 5 or higher, or another C++17-compatible toolchain [1]. +- **Python Environment**: Python 3.10–3.13, preferably managed via Conda or `venv` [1]. +- **Build Tools**: CMake and an optional compiler cache (`ccache`) to significantly speed up recompilation [1]. + +## Available Backends + +ExecuTorch supports multiple backends for Linux, allowing you to optimize execution for specific CPU architectures or dedicated AI accelerators. + +| Backend | Hardware Target | Architecture | Key Features | +|---|---|---|---| +| **XNNPACK** | CPU | x86, x86-64, ARM64 | Highly optimized CPU execution; supports fp32, fp16, and 8-bit quantization; works up to AVX512 on x86-64 [2]. | +| **OpenVINO** | Intel Hardware | x86-64 | Accelerates inference on Intel CPUs, integrated GPUs, discrete GPUs, and NPUs [3]. | +| **Vulkan** | GPU | Cross-platform | Executes on GPUs via GLSL compute shaders; primarily focused on Android but supports Linux GPUs with Vulkan 1.1+ [4]. | + +## Building for Linux + +The ExecuTorch CMake build system includes a dedicated `linux` preset that configures the runtime with the features and backends common for Linux targets [5]. + +### 1. Environment Setup + +Begin by cloning the ExecuTorch repository and configuring your Python environment. Once the environment is active, install the Python dependencies and the default backends (which includes XNNPACK). + +```bash +# Clone and setup the environment +git clone -b viable/strict https://github.com/pytorch/executorch.git +cd executorch +conda create -yn executorch python=3.10.0 +conda activate executorch + +# Install Python packages and dependencies +./install_executorch.sh +``` + +### 2. Compile the Runtime + +With the environment configured, use the `linux` CMake preset to build the C++ runtime. This process will compile the core ExecuTorch libraries and the registered backends. + +```bash +mkdir cmake-out +cmake -B cmake-out --preset linux +cmake --build cmake-out -j10 +``` + +This will generate the `libexecutorch.a` static library and the associated backend libraries (e.g., `libxnnpack_backend.a`). + +### 3. OpenVINO Integration (Optional) + +If you intend to target Intel hardware, the OpenVINO backend requires additional setup. You must install the OpenVINO toolkit and build the backend separately using the provided scripts. + +```bash +# From the executorch/backends/openvino/ directory +pip install -r requirements.txt +cd scripts/ +./openvino_build.sh +``` + +This generates `libopenvino_backend.a` in the `cmake-out/backends/openvino/` directory [3]. + +## Runtime Integration + +To integrate ExecuTorch into your Linux C++ application, link against the compiled runtime and backend libraries. + +When linking the XNNPACK backend, the use of static initializers requires linking with the whole-archive flag to ensure the backend registration code is not stripped by the linker [2]. + +```cmake +# CMakeLists.txt +add_subdirectory("executorch") + +target_link_libraries( + my_linux_app + PRIVATE + executorch + extension_module_static + extension_tensor + optimized_native_cpu_ops_lib + $ +) +``` + +No additional code is required to initialize the backends; any `.pte` file exported for XNNPACK or OpenVINO will automatically execute on the appropriate hardware when loaded by the `Module` API. + +## Next Steps + +- **{doc}`backends/xnnpack/xnnpack-overview`** — Deep dive into XNNPACK export and execution. +- **{doc}`build-run-openvino`** — Deep dive into OpenVINO setup and hardware acceleration. +- **{doc}`using-executorch-cpp`** — Learn how to use the C++ `Module` API to load and run models. + +--- + +## References + +[1] ExecuTorch Documentation: [System Requirements](using-executorch-building-from-source.md#system-requirements) +[2] ExecuTorch Documentation: [XNNPACK Backend](backends/xnnpack/xnnpack-overview.md) +[3] ExecuTorch Documentation: [OpenVINO Backend](build-run-openvino.md) +[4] ExecuTorch Documentation: [Vulkan Backend](backends/vulkan/vulkan-overview.md) +[5] ExecuTorch Documentation: [Building the C++ Runtime](using-executorch-building-from-source.md#building-the-c-runtime) diff --git a/docs/source/desktop-macos.md b/docs/source/desktop-macos.md new file mode 100644 index 00000000000..ccf72301383 --- /dev/null +++ b/docs/source/desktop-macos.md @@ -0,0 +1,95 @@ +# macOS Desktop Deployment + +ExecuTorch provides robust support for macOS deployment, offering hardware-accelerated execution across both Apple Silicon and Intel-based Macs. The runtime is optimized to take advantage of Apple's Core ML framework, Metal Performance Shaders (MPS), and the CPU-optimized XNNPACK backend. + +This guide covers the platform-specific requirements, available backends, and steps to build and run ExecuTorch natively on macOS. + +## Prerequisites + +To build and run ExecuTorch on macOS, ensure your system meets the following minimum requirements: + +- **Operating System**: macOS Big Sur (11.0) or higher. For Core ML and MPS support, macOS 13.0+ and 12.4+ are required, respectively [1]. +- **Development Tools**: Xcode 14.1 or higher. The Xcode Command Line Tools must be installed (`xcode-select --install`) [2]. +- **Python Environment**: Python 3.10–3.13, preferably managed via Conda or `venv` [3]. + +### Intel Mac Considerations + +For Intel-based macOS systems, PyTorch does not provide pre-built binaries. When installing ExecuTorch Python dependencies, you must build PyTorch from source by passing specific flags to the installation script: + +```bash +./install_executorch.sh --use-pt-pinned-commit --minimal +``` + +## Available Backends + +ExecuTorch supports three primary backends for macOS, allowing you to target the CPU, GPU, or Apple Neural Engine (ANE) depending on your hardware and model requirements. + +| Backend | Hardware Target | Minimum macOS | Key Features | +|---|---|---|---| +| **Core ML** | CPU, GPU, ANE | 13.0 | Dynamic dispatch across all Apple hardware; supports fp32 and fp16; recommended for Apple Silicon [2]. | +| **MPS** | Apple Silicon GPU | 12.4 | Direct execution on Metal Performance Shaders; supports fp32 and fp16 [4]. | +| **XNNPACK** | CPU (ARM64 & x86_64) | 11.0 | Highly optimized CPU execution; supports 8-bit quantization; works on both Apple Silicon and Intel Macs [5]. | + +## Building for macOS + +The ExecuTorch CMake build system includes a dedicated `macos` preset that configures the runtime with the features and backends common for Mac targets [3]. + +### 1. Enable Required Backends + +By default, the ExecuTorch installation script builds the XNNPACK and Core ML backends. If you intend to use the MPS backend, you must enable it during the initial setup: + +```bash +CMAKE_ARGS="-DEXECUTORCH_BUILD_MPS=ON" ./install_executorch.sh +``` + +### 2. Compile the Runtime + +Once the Python environment is configured, use the `macos` CMake preset to build the C++ runtime: + +```bash +mkdir cmake-out +cmake -B cmake-out --preset macos +cmake --build cmake-out -j10 +``` + +This will compile the core ExecuTorch libraries and the registered backends (e.g., `libxnnpack_backend.a`, `libcoremldelegate.a`). + +## Runtime Integration + +To integrate ExecuTorch into your macOS C++ application, link against the compiled runtime and backend libraries. + +When linking the Core ML or XNNPACK backends, the use of static initializers requires linking with the whole-archive flag to ensure the backend registration code is not stripped by the linker [2] [5]. + +```cmake +# CMakeLists.txt +add_subdirectory("executorch") + +target_link_libraries( + my_macos_app + PRIVATE + executorch + extension_module_static + extension_tensor + optimized_native_cpu_ops_lib + $ + $ +) +``` + +No additional code is required to initialize the backends; any `.pte` file exported for Core ML, MPS, or XNNPACK will automatically execute on the appropriate hardware when loaded by the `Module` API. + +## Next Steps + +- **{doc}`backends/coreml/coreml-overview`** — Deep dive into Core ML export and execution. +- **{doc}`backends/mps/mps-overview`** — Deep dive into MPS export and execution. +- **{doc}`using-executorch-cpp`** — Learn how to use the C++ `Module` API to load and run models. + +--- + +## References + +[1] ExecuTorch Documentation: [Building from Source](using-executorch-building-from-source.md) +[2] ExecuTorch Documentation: [Core ML Backend](backends/coreml/coreml-overview.md) +[3] ExecuTorch Documentation: [Building the C++ Runtime](using-executorch-building-from-source.md#building-the-c-runtime) +[4] ExecuTorch Documentation: [MPS Backend](backends/mps/mps-overview.md) +[5] ExecuTorch Documentation: [XNNPACK Backend](backends/xnnpack/xnnpack-overview.md) diff --git a/docs/source/desktop-mps.md b/docs/source/desktop-mps.md deleted file mode 100644 index d6f305d33aa..00000000000 --- a/docs/source/desktop-mps.md +++ /dev/null @@ -1 +0,0 @@ -```{include} backends-mps.md diff --git a/docs/source/desktop-openvino.md b/docs/source/desktop-openvino.md deleted file mode 100644 index a0fd5774c73..00000000000 --- a/docs/source/desktop-openvino.md +++ /dev/null @@ -1 +0,0 @@ -```{include} build-run-openvino.md diff --git a/docs/source/desktop-section.md b/docs/source/desktop-section.md index bf306e7c43b..377bdd8ba8b 100644 --- a/docs/source/desktop-section.md +++ b/docs/source/desktop-section.md @@ -1,24 +1,65 @@ (desktop-section)= + # Desktop & Laptop Platforms -Deploy ExecuTorch on Linux, macOS, and Windows with optimized backends for each platform. +ExecuTorch provides robust, high-performance deployment capabilities for desktop and laptop environments across macOS, Linux, and Windows. By leveraging native hardware acceleration and a cross-platform C++ runtime, developers can execute PyTorch models efficiently on CPUs, GPUs, and dedicated AI accelerators (NPUs/ANEs). + +This section provides comprehensive, platform-specific guides for setting up, building, and optimizing ExecuTorch for native desktop execution. + +## Platform-Specific Guides + +Select your target operating system below for detailed setup instructions, prerequisites, and backend integration steps. + +::::{grid} 3 +:::{grid-item-card} macOS +:class-card: card-prerequisites +**→ {doc}`desktop-macos`** + +Native execution on Apple Silicon and Intel Macs using Core ML, MPS, and XNNPACK. +::: +:::{grid-item-card} Linux +:class-card: card-prerequisites +**→ {doc}`desktop-linux`** -## Platform Overview & Runtime +High-performance deployment on Linux distributions using XNNPACK and OpenVINO. +::: +:::{grid-item-card} Windows +:class-card: card-prerequisites +**→ {doc}`desktop-windows`** -- {doc}`using-executorch-cpp` — C++ runtime integration guide -- {doc}`using-executorch-building-from-source` — Building ExecuTorch from source +Native Windows and WSL support using XNNPACK and OpenVINO with Visual Studio. +::: +:::: -## Backends +## Backend Hardware Support -- {doc}`desktop-backends` — Available desktop backends and platform-specific optimization +ExecuTorch relies on specialized backends to map model execution to the underlying desktop hardware. The table below summarizes the available backends and their supported platforms. + +| Backend | Primary Hardware Target | macOS | Linux | Windows | Key Features | +|---|---|:---:|:---:|:---:|---| +| **[XNNPACK](backends/xnnpack/xnnpack-overview)** | CPU (ARM64, x86-64) | ✅ | ✅ | ✅ | Highly optimized CPU execution; supports fp32, fp16, and 8-bit quantization. Included by default. | +| **[Core ML](backends/coreml/coreml-overview)** | Apple CPU, GPU, ANE | ✅ | ❌ | ❌ | Dynamic dispatch across Apple hardware; recommended for Apple Silicon. | +| **[MPS](backends/mps/mps-overview)** | Apple Silicon GPU | ✅ | ❌ | ❌ | Direct execution on Metal Performance Shaders for high-throughput GPU inference. | +| **[OpenVINO](build-run-openvino)** | Intel CPU, GPU, NPU | ❌ | ✅ | ✅ | Intel-optimized execution across integrated graphics, discrete GPUs, and NPUs. | +| **[Vulkan](backends/vulkan/vulkan-overview)** | Cross-platform GPU | ❌ | ✅ | ❌ | GPU execution via GLSL compute shaders; primarily focused on Android but supports Linux. | + +## Core Runtime Integration + +Regardless of the target desktop platform, integrating ExecuTorch into a native application follows a consistent pattern using the C++ `Module` API. + +- **{doc}`using-executorch-cpp`** — Learn how to use the C++ `Module` API to load `.pte` files, configure memory allocation, and execute inferences natively. +- **{doc}`using-executorch-building-from-source`** — Comprehensive reference for the CMake build system, configuration options, and presets used across all desktop platforms. ## Tutorials -- {doc}`raspberry_pi_llama_tutorial` — Cross compiling ExecuTorch for the Raspberry Pi on Linux Host +- **{doc}`raspberry_pi_llama_tutorial`** — Cross compiling ExecuTorch for the Raspberry Pi on a Linux Host. ```{toctree} :hidden: -using-executorch-cpp -using-executorch-building-from-source -desktop-backends -raspberry_pi_llama_tutorial +:maxdepth: 2 +:caption: Desktop Platforms + +desktop-macos +desktop-linux +desktop-windows +``` diff --git a/docs/source/desktop-windows.md b/docs/source/desktop-windows.md new file mode 100644 index 00000000000..d51f809559d --- /dev/null +++ b/docs/source/desktop-windows.md @@ -0,0 +1,117 @@ +# Windows Desktop Deployment + +ExecuTorch provides support for native Windows deployment, enabling high-performance model execution across a wide range of hardware configurations. The runtime leverages optimized backends like XNNPACK for CPU execution and OpenVINO for Intel hardware acceleration. + +This guide details the system requirements, available backends, and steps to build and run ExecuTorch natively on Windows operating systems. + +## Prerequisites + +ExecuTorch is actively tested and supported on Windows 10 and newer. Ensure your environment meets the following minimum requirements: + +- **Operating System**: Windows 10 or higher [1]. +- **Compiler Toolchain**: Visual Studio 2022+ with Clang-CL support. Clang/LLVM support must be explicitly enabled in the Visual Studio installer [1]. +- **Python Environment**: Python 3.10–3.13, preferably managed via Conda or `venv` [1]. +- **Build Tools**: CMake [1]. + +### Windows Subsystem for Linux (WSL) + +As an alternative to native Windows deployment, ExecuTorch fully supports Windows Subsystem for Linux (WSL) with any compatible Linux distribution (e.g., Ubuntu). If using WSL, refer to the **{doc}`desktop-linux`** guide for setup instructions [1]. + +### Important: Enable Symlinks + +ExecuTorch requires symbolic links to be enabled to build the Python components. You must configure Git to support symlinks *before* cloning the repository. Missing symlinks will manifest as an error related to `version.py` when running `pip install .` [1]. + +```bash +# Run this command in an Administrator prompt before cloning +git config --system core.symlinks true +``` + +## Available Backends + +ExecuTorch supports multiple backends for Windows, allowing you to optimize execution for specific CPU architectures or dedicated AI accelerators. + +| Backend | Hardware Target | Architecture | Key Features | +|---|---|---|---| +| **XNNPACK** | CPU | x86, x86-64, ARM64 | Highly optimized CPU execution; supports fp32, fp16, and 8-bit quantization; works up to AVX512 on x86-64 [2]. | +| **OpenVINO** | Intel Hardware | x86-64 | Accelerates inference on Intel CPUs, integrated GPUs, discrete GPUs, and NPUs [3]. | + +## Building for Windows + +The ExecuTorch CMake build system configures the runtime with the features and backends common for Windows targets. + +### 1. Environment Setup + +Begin by cloning the ExecuTorch repository and configuring your Python environment. Once the environment is active, install the Python dependencies and the default backends (which includes XNNPACK). + +```bash +# Clone and setup the environment (ensure symlinks are enabled first) +git clone -b viable/strict https://github.com/pytorch/executorch.git +cd executorch +conda create -yn executorch python=3.10.0 +conda activate executorch + +# Install Python packages and dependencies +./install_executorch.sh +``` + +### 2. Compile the Runtime + +With the environment configured, use CMake to build the C++ runtime. This process will compile the core ExecuTorch libraries and the registered backends. + +```bash +mkdir cmake-out +cmake -B cmake-out +cmake --build cmake-out -j10 +``` + +This will generate the `executorch.lib` static library and the associated backend libraries (e.g., `xnnpack_backend.lib`). + +### 3. OpenVINO Integration (Optional) + +If you intend to target Intel hardware, the OpenVINO backend requires additional setup. You must install the OpenVINO toolkit and build the backend separately using the provided scripts. + +```bash +# From the executorch/backends/openvino/ directory +pip install -r requirements.txt +cd scripts/ +./openvino_build.sh +``` + +This generates the OpenVINO backend library in the `cmake-out/backends/openvino/` directory [3]. + +## Runtime Integration + +To integrate ExecuTorch into your Windows C++ application, link against the compiled runtime and backend libraries. + +When linking the XNNPACK backend, the use of static initializers requires linking with the whole-archive flag to ensure the backend registration code is not stripped by the linker [2]. + +```cmake +# CMakeLists.txt +add_subdirectory("executorch") + +target_link_libraries( + my_windows_app + PRIVATE + executorch + extension_module_static + extension_tensor + optimized_native_cpu_ops_lib + $ +) +``` + +No additional code is required to initialize the backends; any `.pte` file exported for XNNPACK or OpenVINO will automatically execute on the appropriate hardware when loaded by the `Module` API. + +## Next Steps + +- **{doc}`backends/xnnpack/xnnpack-overview`** — Deep dive into XNNPACK export and execution. +- **{doc}`build-run-openvino`** — Deep dive into OpenVINO setup and hardware acceleration. +- **{doc}`using-executorch-cpp`** — Learn how to use the C++ `Module` API to load and run models. + +--- + +## References + +[1] ExecuTorch Documentation: [System Requirements](using-executorch-building-from-source.md#system-requirements) +[2] ExecuTorch Documentation: [XNNPACK Backend](backends/xnnpack/xnnpack-overview.md) +[3] ExecuTorch Documentation: [OpenVINO Backend](build-run-openvino.md) diff --git a/docs/source/desktop-xnnpack.md b/docs/source/desktop-xnnpack.md deleted file mode 100644 index 4a85dec946b..00000000000 --- a/docs/source/desktop-xnnpack.md +++ /dev/null @@ -1 +0,0 @@ -```{include} backends/xnnpack/xnnpack-overview.md diff --git a/docs/source/edge-platforms-section.md b/docs/source/edge-platforms-section.md index 209986507fa..04672929c00 100644 --- a/docs/source/edge-platforms-section.md +++ b/docs/source/edge-platforms-section.md @@ -30,14 +30,15 @@ Key features: ## Desktop & Laptop Platforms -Deploy ExecuTorch on Linux, macOS, and Windows with optimized backends. +Deploy ExecuTorch natively on macOS, Linux, and Windows with platform-specific hardware acceleration. -**→ {doc}`desktop-section` — Complete desktop deployment guide** +**→ {doc}`desktop-section` — Desktop deployment guide** Key features: -- Cross-platform C++ runtime -- Platform-specific optimization (OpenVINO, CoreML, MPS) -- CPU and GPU acceleration options + +- Platform-specific guides for macOS (Core ML, MPS, XNNPACK), Linux (XNNPACK, OpenVINO), and Windows (XNNPACK, OpenVINO) +- Cross-platform C++ runtime with CMake build presets for each OS +- CPU and GPU acceleration, including Apple Neural Engine and Intel NPU support ## Embedded Systems diff --git a/docs/source/platforms-desktop.md b/docs/source/platforms-desktop.md deleted file mode 100644 index ba22786576f..00000000000 --- a/docs/source/platforms-desktop.md +++ /dev/null @@ -1,23 +0,0 @@ -# Desktop & Laptop - -ExecuTorch supports desktop and laptop deployment across Linux, macOS, and Windows. - -## Platform-Specific Guides -- [C++ Runtime Integration](using-executorch-cpp) - Complete setup guide -- [Building from Source](using-executorch-building-from-source) - -## Available Backends by Platform - -### Linux -- [XNNPACK (CPU)](backends/xnnpack/xnnpack-overview.md) -- [OpenVINO (Intel)](build-run-openvino) -- [ARM Ethos-U (ARM64)](backends-arm-ethos-u) - -### macOS -- [CoreML (recommended)](backends-coreml) -- [MPS (Apple Silicon)](backends-mps) -- [XNNPACK (CPU)](backends/xnnpack/xnnpack-overview.md) - -### Windows -- [XNNPACK (CPU)](backends/xnnpack/xnnpack-overview.md) -- [OpenVINO (Intel)](build-run-openvino)