Skip to content

executorch save + load#4169

Open
lanluo-nvidia wants to merge 22 commits intomainfrom
lluo/executorch_load
Open

executorch save + load#4169
lanluo-nvidia wants to merge 22 commits intomainfrom
lluo/executorch_load

Conversation

@lanluo-nvidia
Copy link
Copy Markdown
Collaborator

@lanluo-nvidia lanluo-nvidia commented Apr 8, 2026

Description

executorch local build / save / load workflow:

  1. build libexecutorch_core.a with ExecuTorch source tree
export EXECUTORCH_ROOT=/home/lanl/git/executorch
 export CMAKE_PREFIX_PATH=/home/lanl/miniconda3/envs/torch_tensorrt_py310/lib/python3.10/site-packages/torch/
 share/cmake

 cmake -S "${EXECUTORCH_ROOT}" -B "${EXECUTORCH_ROOT}/cmake-out" \
   -DCMAKE_BUILD_TYPE=Release \
   -DBUILD_TESTING=OFF \
   -DEXECUTORCH_BUILD_PYBIND=OFF \
   -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
   -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
   -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON \
   -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON

 cmake --build "${EXECUTORCH_ROOT}/cmake-out" --target executorch_core -j
  1. build executorch TensorRT backend static library libexecutorch_trt_backend.a
  cmake -S . -B build-executorch \
    -DBUILD_TORCHTRT_EXECUTORCH=ON \
    -DEXECUTORCH_ROOT="${EXECUTORCH_ROOT}" \
    -DEXECUTORCH_CORE_LIBRARY="${EXECUTORCH_ROOT}/cmake-out/libexecutorch_core.a" \
    -DCMAKE_PREFIX_PATH="${CMAKE_PREFIX_PATH}"

  cmake --build build-executorch --target executorch_trt_backend -j
  1. build the executor runner app
    cmake --build build-executorch --target trt_executor_runner -j

  2. save the model in executorch format
    this step require executorch, torch, tensorrt, torch_tensorrt wheel to be installed
    python examples/torchtrt_executorch_example/export_static_shape.py --model_path=model.pte

  3. load the model saved in executorch format
    this step does not require torch_tensorrt wheel, only the executor runner app binary, tensorrt, torch, executorch.
    ./build-executorch/bin/trt_executor_runner --model_path=model.pte

bazel release tar ball package workflow:

export EXECUTORCH_ROOT=/home/lanl/git/executorch
bazel build //:libtorchtrt_executorch --compilation_mode opt --config=linux

Expected output:

  • bazel-bin/libtorchtrt_executorch.tar.gz

That tarball now includes:

  • torch_tensorrt/lib/libtorchtrt.so
  • torch_tensorrt/lib/libtorchtrt_runtime.so
  • torch_tensorrt/lib/libtorchtrt_plugins.so
  • torch_tensorrt/lib/libexecutorch_core.a
  • torch_tensorrt/lib/libexecutorch_trt_backend.a

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@meta-cla meta-cla Bot added the cla signed label Apr 8, 2026
@github-actions github-actions Bot added component: core Issues re: The core compiler component: build system Issues re: Build system component: api [Python] Issues re: Python API component: runtime component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 8, 2026
@github-actions github-actions Bot requested a review from narendasan April 8, 2026 02:21
@lanluo-nvidia lanluo-nvidia changed the title [WIP] executorch save + load [WIP] [Not ready for review] executorch save + load Apr 8, 2026
@github-actions github-actions Bot added the component: api [C++] Issues re: C++ API label Apr 9, 2026
@lanluo-nvidia lanluo-nvidia marked this pull request as ready for review April 9, 2026 21:57
@lanluo-nvidia lanluo-nvidia changed the title [WIP] [Not ready for review] executorch save + load executorch save + load Apr 9, 2026
@github-actions github-actions Bot added the component: tests Issues re: Tests label Apr 23, 2026
Comment on lines +29 to +59
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.pick.outputs.matrix }}
steps:
- id: pick
env:
FULL_MATRIX: ${{ inputs.build-matrix }}
run: |
set -euo pipefail
python - <<'PY' >> "${GITHUB_OUTPUT}"
import json
import os

matrix = json.loads(os.environ["FULL_MATRIX"])
include = matrix.get("include", [])
if not include:
raise SystemExit("build-matrix include[] is empty")

preferred = None
for entry in include:
if entry.get("python_version") == "3.11":
preferred = entry
break

if preferred is None:
preferred = include[0]

print("matrix=" + json.dumps({"include": [preferred]}))
PY

build:
Comment on lines +60 to +89
needs: select-matrix
uses: ./.github/workflows/linux-test.yml
with:
job-name: executorch-static-build
repository: ${{ inputs.repository }}
ref: ${{ inputs.ref }}
test-infra-repository: ${{ inputs.test-infra-repository }}
test-infra-ref: ${{ inputs.test-infra-ref }}
build-matrix: ${{ needs.select-matrix.outputs.matrix }}
script: |
set -euo pipefail
EXECUTORCH_SRC="${RUNNER_TEMP}/executorch"
EXECUTORCH_BUILD="${EXECUTORCH_SRC}/cmake-out"

git clone --depth 1 https://github.com/pytorch/executorch.git "${EXECUTORCH_SRC}"
cmake -S "${EXECUTORCH_SRC}" -B "${EXECUTORCH_BUILD}" \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TESTING=OFF \
-DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=OFF \
-DEXECUTORCH_BUILD_PYBIND=OFF \
-DEXECUTORCH_BUILD_PORTABLE_OPS=OFF
cmake --build "${EXECUTORCH_BUILD}" --target executorch_core -j"$(nproc)"

TORCH_CMAKE_PREFIX="$(python3 -c 'import torch; print(torch.utils.cmake_prefix_path)')"
cmake -S . -B build-executorch \
-DCMAKE_PREFIX_PATH="${TORCH_CMAKE_PREFIX}" \
-DBUILD_TORCHTRT_EXECUTORCH=ON \
-DEXECUTORCH_ROOT="${EXECUTORCH_SRC}" \
-DEXECUTORCH_CORE_LIBRARY="${EXECUTORCH_BUILD}/libexecutorch_core.a"
cmake --build build-executorch --target trt_executor_runner -j"$(nproc)"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [C++] Issues re: C++ API component: api [Python] Issues re: Python API component: build system Issues re: Build system component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: runtime component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants