From 7a78aac17e4cc693da14b64cabe95b54035ce094 Mon Sep 17 00:00:00 2001 From: tlopex <820958424@qq.com> Date: Mon, 25 May 2026 16:36:37 -0400 Subject: [PATCH 1/2] finish1 --- docs/contribute/code_guide.rst | 2 + docs/contribute/index.rst | 1 + .../testing.rst} | 49 ++-- docs/errors.rst | 108 ++++---- docs/how_to/dev/index.rst | 28 -- docs/how_to/dev/setup_rpc_system.rst | 243 ------------------ .../tutorials/cross_compilation_and_rpc.py | 189 +++++++++++++- docs/index.rst | 7 +- 8 files changed, 280 insertions(+), 347 deletions(-) rename docs/{how_to/dev/pytest_target_parametrization.rst => contribute/testing.rst} (90%) delete mode 100644 docs/how_to/dev/index.rst delete mode 100644 docs/how_to/dev/setup_rpc_system.rst diff --git a/docs/contribute/code_guide.rst b/docs/contribute/code_guide.rst index 28d61c505359..fd40cec579cf 100644 --- a/docs/contribute/code_guide.rst +++ b/docs/contribute/code_guide.rst @@ -128,6 +128,8 @@ Python Code Styles Writing Python Tests -------------------- We use `pytest `_ for all python testing. ``tests/python`` contains all the tests. +See :doc:`testing` for details on running tests, target parametrization, +and the target-specific marks used by CI. If you want your test to run over a variety of targets, use the :py:func:`tvm.testing.parametrize_targets` decorator. For example: diff --git a/docs/contribute/index.rst b/docs/contribute/index.rst index d30dd3e8b0b4..eec404fe9022 100644 --- a/docs/contribute/index.rst +++ b/docs/contribute/index.rst @@ -46,6 +46,7 @@ Here are guidelines for contributing to various aspect of the project: committer_guide document code_guide + testing git_howto ci release_process diff --git a/docs/how_to/dev/pytest_target_parametrization.rst b/docs/contribute/testing.rst similarity index 90% rename from docs/how_to/dev/pytest_target_parametrization.rst rename to docs/contribute/testing.rst index 165d8ac0a59c..c2f502503099 100644 --- a/docs/how_to/dev/pytest_target_parametrization.rst +++ b/docs/contribute/testing.rst @@ -15,11 +15,17 @@ specific language governing permissions and limitations under the License. +Testing TVM +=========== + +This page describes how to write and run Python tests for TVM, +including the target parametrization utilities used by CI. + Python Target Parametrization -============================= +----------------------------- Summary -------- +~~~~~~~ For any supported runtime, TVM should produce numerically correct results. Therefore, when writing unit tests that validate @@ -28,7 +34,7 @@ runtimes. Since this is a very common use case, TVM has helper functions to parametrize unit tests such that they will run on all targets that are enabled and have a compatible device. -A single python function in the test suite can expand to several +A single Python function in the test suite can expand to several parameterized unit tests, each of which tests a single target device. In order for a test to be run, all of the following must be true. @@ -47,7 +53,7 @@ In order for a test to be run, all of the following must be true. runtime. Unit-Test File Contents ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ .. _pytest-marks: https://docs.pytest.org/en/stable/how-to/mark.html @@ -160,27 +166,28 @@ listed as skipped. There also exists a ``tvm.testing.enabled_targets()`` that returns all targets that are enabled and runnable on the current machine, based on the environment variable ``TVM_TEST_TARGETS``, the build -configuration, and the physical hardware present. Most current tests +configuration, and the physical hardware present. Some legacy tests explicitly loop over the targets returned from ``enabled_targets()``, -but it should not be used for new tests. The pytest output for this -style silently skips runtimes that are disabled in ``config.cmake``, -or do not have a device on which they can run. In addition, the test -halts on the first target to fail, which is ambiguous as to whether -the error occurs on a particular target, or on every target. +but this style should not be used for new tests. The pytest output +for this style silently skips runtimes that are disabled in +``config.cmake``, or do not have a device on which they can run. In +addition, the test halts on the first target to fail, which is +ambiguous as to whether the error occurs on a particular target, or on +every target. .. code-block:: python # Old style, do not use. def test_function(): - for target,dev in tvm.testing.enabled_targets(): + for target, dev in tvm.testing.enabled_targets(): # Test code goes here -Running locally ---------------- +Running Locally +~~~~~~~~~~~~~~~ -To run the python unit-tests locally, use the command ``pytest`` in +To run the Python unit tests locally, use the command ``pytest`` in the ``${TVM_HOME}`` directory. - Environment variables @@ -206,19 +213,19 @@ the ``${TVM_HOME}`` directory. system without a specific backend installed. - The ``-m`` argument only runs unit tests that are tagged with a - specific pytest marker. The most frequent usage is to use ``m - gpu`` to run only tests that are marked with + specific pytest marker. The most frequent usage is to use + ``-m gpu`` to run only tests that are marked with ``@pytest.mark.gpu`` and use a GPU to run. It can also be used - to run only tests that do not use a GPU, by passing ``m 'not - gpu'``. + to run only tests that do not use a GPU, by passing ``not gpu`` + as the marker expression to ``-m``. Note: This filtering takes place after the selection of targets based on the ``TVM_TEST_TARGETS`` environment variable. Even if ``-m gpu`` is specified, if ``TVM_TEST_TARGETS`` does not contain GPU targets, no GPU tests will be run. -Running in local docker container ---------------------------------- +Running in a Local Docker Container +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. _tlcpack: https://hub.docker.com/u/tlcpack @@ -243,7 +250,7 @@ and make a symlink from ``build`` to the appropriate folder when entering/exiting docker. Running in CI -------------- +~~~~~~~~~~~~~ Everything in the CI starts from the task definitions present in the Jenkinsfile. This includes defining which docker image gets used, diff --git a/docs/errors.rst b/docs/errors.rst index 4d9829502c63..3cc43461634e 100644 --- a/docs/errors.rst +++ b/docs/errors.rst @@ -16,56 +16,58 @@ under the License. -Handle TVM Errors -================= - -When running TVM, you may encounter an error message like: - -.. code:: - - --------------------------------------------------------------- - An error occurred during the execution of TVM. - For more information, please see: https://tvm.apache.org/docs/errors.html - --------------------------------------------------------------- - -Congratulations! You found this page. Below are some hints on how to interpret -these error messages and what you can do when they occur. - -Where do these errors come from? --------------------------------- - -This error is caused by an internal invariant being violated during TVM's -execution. On a technical level, the message is generated by the -``TVM_FFI_ICHECK`` macro, found in ``include/tvm/runtime/logging.h``. -The ``TVM_FFI_ICHECK`` macro is used in many places in the TVM code to assert -some condition is true during execution; any time the assertion fails, TVM -will exit with the error message shown above. - -For more details about how errors are handled and generated by TVM, please -see :ref:`error-handling-guide`. - -What should I do when I encounter such an error? ------------------------------------------------- - -First of all, *don't panic*. Well, you can panic, but it won't help. - -The best course of action is to search the -`Apache TVM Discuss Forum `_ -for the error you are encountering, to see if this has been a problem -that others have encountered, and what the solution might be. -If this error is the result of a bug that has been fixed in a more -recent version of TVM, you may need to update to a newer version. - -If you do not find an existing Discuss Forum thread about your -issue, you are welcome to start a new thread on the forum with details -on the problem. *Please* include in your posting the following key -pieces of information: - -* The version of TVM you are using (e.g., the git commit hash of your source tree). -* Which hardware and operating system version you are running TVM on. -* Which hardware device and OS you are targeting for your TVM compilation. -* Details on the model, inputs, or other information about the workload, which can - be used to reproduce your problem. - -Without these details it is very difficult for the TVM developers to do very -much to help you. +Troubleshooting TVM Errors +========================== + +TVM may raise errors from Python code, from C++ code reached through the +FFI, or from generated runtime modules. Error messages usually include +a Python stack trace, and may also include a C++ stack trace when the +error crosses the TVM FFI boundary. + +Some errors report invalid user input, unsupported operators, missing +runtime features, or unavailable hardware. Others report a failed +internal check, usually raised by ``TVM_FFI_ICHECK`` or +``TVM_FFI_THROW`` in C++ code. Internal check failures often indicate +that TVM reached a state that the implementation did not expect. + +What to Check First +------------------- + +- Make sure the TVM Python package and native libraries come from the + same build. A common symptom of a mismatched environment is importing + Python files from one checkout while loading ``libtvm`` from another. +- Check that the required runtime is enabled in ``config.cmake``. For + example, CUDA tests and CUDA compilation require a TVM build with + CUDA support enabled. +- Check that the target hardware is available to the process. GPU + tests may be skipped or fail if the device is not visible inside the + current container or environment. +- If the error occurs while importing or converting a model, reduce the + input to the smallest model, operator, or shape that reproduces the + issue. + +Reporting an Issue +------------------ + +Search the `Apache TVM Discuss Forum `_ +and the `TVM issue tracker `_ +for the exact error message first. If you do not find an existing +report, include the following details when starting a new discussion or +filing an issue: + +- The TVM version or git commit hash. +- The Python version, operating system, and hardware. +- The target and runtime being used, such as LLVM, CUDA, Vulkan, or RPC. +- The relevant build configuration from ``config.cmake``. +- A minimal script, model, input shape, or IR module that reproduces the + failure. +- The full error message, including both Python and C++ stack traces + when present. + +Developer Notes +--------------- + +For guidance on raising typed errors from TVM code, see +:ref:`error-handling-guide`. That guide covers when to use specific +error types, how C++ error prefixes map to Python exceptions, and how +``TVM_FFI_ICHECK`` interacts with TVM's error handling. diff --git a/docs/how_to/dev/index.rst b/docs/how_to/dev/index.rst deleted file mode 100644 index c815871b4147..000000000000 --- a/docs/how_to/dev/index.rst +++ /dev/null @@ -1,28 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - -Development Guides -================== -This section contains a collection of tips about how to work on -various areas of the TVM stack. - -.. toctree:: - :maxdepth: 1 - - pytest_target_parametrization - setup_rpc_system - ../../errors diff --git a/docs/how_to/dev/setup_rpc_system.rst b/docs/how_to/dev/setup_rpc_system.rst deleted file mode 100644 index f5d6e99a30ae..000000000000 --- a/docs/how_to/dev/setup_rpc_system.rst +++ /dev/null @@ -1,243 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - -Setup RPC System -================ - -Remote procedure call (RPC) is a very important and useful feature of Apache TVM, it allows us to run compiled Neural Network (NN) models on the real hardware without need to touch the remote device, the output result will be passed back automatically through network. - -By eliminating the manual work like, dumping input data to file, copying the exported NN model to remote device, setuping the device user environment, copying the output result to host development environment, RPC improve the development efficiency extremely. - -In addition, because only the execution part of the compiled NN model is run on the remote device, all other parts are run on host development environment, so any Python packages can be used to do the preprocess and postprocess works. - -RPC is very helpful in below 2 situations - -- **Hardware resources are limited** - - RPC’s queue and resource management mechanism can make the hardware devices serve many developers and test jobs to run the compiled NN models correctly. - -- **Early-stage end to end evaluation** - - Except the compiled NN model, all other parts are executed on the host development environment, so the complex preprocess or postprocess can be implemented easily. - - -Suggested Architecture ----------------------- - -Apache TVM RPC contains 3 tools, RPC tracker, RPC proxy, and PRC server. The RPC server is the necessary one, an RPC system can work correctly without RPC proxy and RPC tracker. RPC proxy is needed when you can’t access the RPC server directly. RPC tracker is strongly suggested to be added in your RPC system, because it provides many useful features, e.g., queue capability, multiple RPC servers management, manage RPC server through key instead of IP address. - -.. figure:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/dev/how-to/rpc_system_suggested_arch.svg - :align: center - :width: 85% - -As above figure shown, because there aren’t physical connection channels between machine A and machine C, D, so we set up a RPC proxy on machine B. The RPC tracker manage a request queue per RPC key, each user can request an RPC server from RPC tracker by a RPC key at anytime, if there is a idle RPC server with the same RPC key, then RPC tracker assign the RPC server to the user, if there isn’t a idle RPC server for the moment, the request will be put into the request queue of that RPC key, and check for it later. - - -Setup RPC Tracker and RPC Proxy -------------------------------- - -In general, RPC tracker and RPC proxy only need to be run on host machine, e.g., development server or PC, they needn't depend on any environment of device machine, so the only work need to do for setting up them is executing below commands on the corresponding machine after installing Apache TVM according to the official document ``_. - -- RPC Tracker - - .. code-block:: shell - - $ python3 -m tvm.exec.rpc_tracker --host RPC_TRACKER_IP --port 9190 --port-end 9191 - - -- RPC Proxy - - .. code-block:: shell - - $ python3 -m tvm.exec.rpc_proxy --host RPC_PROXY_IP --port 9090 --port-end 9091 --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT - - -Please modify the *RPC_TRACKER_IP*, *RPC_TRACKER_PORT*, *RPC_PROXY_IP*, and the port numbers in above commands according to your concrete environment, the option ``port-end`` can be used to avoid the service start with an unexpected port number, which may cause other service can't be connected correctly, this is important especially for auto testing system. - - -Setup RPC Server ----------------- - -In our community, there is multiple RPC server implementations, e.g., ``apps/android_rpc``, ``apps/cpp_rpc``, ``apps/ios_rpc``, below content only focus on the Python version RPC server which is implemented by ``python/tvm/exec/rpc_server.py``, for the setup instruction of other version RPC server please refer to the document of its corresponding directory. - -RPC server need to be run on device machine, and it usually will depend on xPU driver, the enhanced TVM runtime with xPU support, and other libraries, so please setup the dependent components first, e.g., install the KMD driver, ensure the required dynamic libraries can be found from environment variable ``LD_LIBRARY_PATH``. - -If the required compilation environment can be setup on your device machine, i.e., you needn't to do the cross compilation, then just follow the instruction of ``_ to compile the TVM runtime and directly jump to the step :ref:`launch-rpc-server`. - -1. Cross Compile TVM Runtime -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -We use CMake to manage the compile process, for cross compilation, CMake need a toolchain file to get the required information, so you need to prepare this file according to your device platform, below is a example for the device machine which CPU is 64bit ARM architecture and the operating system is Linux. - -.. code-block:: cmake - - set(CMAKE_SYSTEM_NAME Linux) - set(root_dir "/XXX/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu") - - set(CMAKE_C_COMPILER "${root_dir}/bin/aarch64-linux-gnu-gcc") - set(CMAKE_CXX_COMPILER "${root_dir}/bin/aarch64-linux-gnu-g++") - set(CMAKE_SYSROOT "${root_dir}/aarch64-linux-gnu/libc") - - set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) - set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) - set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) - set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) - -After executing commands like something below under the root directory of TVM repository, the runtime will be cross compiled successfully, please enable other needed options in file ``config.cmake`` according to your concrete requirement. - -.. code-block:: shell - - $ mkdir cross_build - $ cd cross_build - $ cp ../cmake/config.cmake ./ - - # You maybe need to enable other options, e.g., USE_OPENCL, USE_xPU. - $ sed -i "s|USE_LLVM.*)|USE_LLVM OFF)|" config.cmake - - $ cmake -DCMAKE_TOOLCHAIN_FILE=/YYY/aarch64-linux-gnu.cmake -DCMAKE_BUILD_TYPE=Release .. - $ cmake --build . -j -- runtime - $ cd .. - - -2. Pack and Deploy to Device Machine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Pack the Python version RPC server through the commands like something below. - -.. code-block:: shell - - $ git clean -dxf python - $ cp cross_build/libtvm_runtime.so python/tvm/ - $ tar -czf tvm_runtime.tar.gz python - -Then copy the compress package ``tvm_runtime.tar.gz`` to your concrete device machine, and setting the environment variable ``PYTHONPATH`` correctly through the commands like something below on your device machine. - -.. code-block:: shell - - $ tar -xzf tvm_runtime.tar.gz - $ export PYTHONPATH=`pwd`/python:${PYTHONPATH} - - -.. _launch-rpc-server: - -3. Launch RPC Server -^^^^^^^^^^^^^^^^^^^^ - -The RPC server can be launched on your device machine through the commands like something below, please modify the *RPC_TRACKER_IP*, *RPC_TRACKER_PORT*, *RPC_PROXY_IP*, *RPC_PROXY_PORT*, and *RPC_KEY* according to your concrete environment. - -.. code-block:: shell - - # Use this if you use RPC proxy. - $ python3 -m tvm.exec.rpc_server --host RPC_PROXY_IP --port RPC_PROXY_PORT --through-proxy --key RPC_KEY - # Use this if you needn't use RPC proxy. - $ python3 -m tvm.exec.rpc_server --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT --key RPC_KEY - - -Validate RPC System -------------------- - -.. code-block:: shell - - $ python3 -m tvm.exec.query_rpc_tracker --host RPC_TRACKER_IP --port RPC_TRACKER_PORT - -Through the above command, we can query all available RPC servers and the queue status, if you have 3 RPC servers that connected to the RPC tracker through RPC proxy, the output should be something like below. - -.. code-block:: shell - - Tracker address RPC_TRACKER_IP:RPC_TRACKER_PORT - - Server List - ---------------------------- - server-address key - ---------------------------- - RPC_PROXY_IP:RPC_PROXY_PORT server:proxy[RPC_KEY0,RPC_KEY1,RPC_KEY2] - ---------------------------- - - Queue Status - --------------------------------------- - key total free pending - --------------------------------------- - RPC_KEY0 0 0 3 - --------------------------------------- - - -Troubleshooting ---------------- - -1. The lack of ``numpy`` on device machine caused the RPC server can't be launched. -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The package ``numpy`` is imported in some Python files which RPC server dependent on, and eliminating the import relationship is difficult, for some devices cross compiling ``numpy`` is very hard to do too. - -But actually the TVM runtime doesn't really dependent on ``numpy``, so a very simple workaround is create a dummy ``numpy``, just need to copy the below content into a file named ``numpy.py`` and place it into your Python's ``site-packages`` directory (e.g. ``/usr/local/lib/python3.x/site-packages``). - -.. code-block:: python - - class bool_: - pass - class int8: - pass - class int16: - pass - class int32: - pass - class int64: - pass - class uint8: - pass - class uint16: - pass - class uint32: - pass - class uint64: - pass - class float16: - pass - class float32: - pass - class float64: - pass - class float_: - pass - - class dtype: - def __init__(self, *args, **kwargs): - pass - - class ndarray: - pass - - def sqrt(*args, **kwargs): - pass - - def log(*args, **kwargs): - pass - - def tanh(*args, **kwargs): - pass - - def power(*args, **kwargs): - pass - - def exp(*args, **kwargs): - pass - - -2. The lack of ``cloudpickle`` on device machine caused the RPC server can't be launched. -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Because ``cloudpickle`` package is a pure Python package, so just copying it from other machine to the Python ``site-packages`` directory of the device machine will resolve the problem. diff --git a/docs/how_to/tutorials/cross_compilation_and_rpc.py b/docs/how_to/tutorials/cross_compilation_and_rpc.py index 7ef45c38b0e3..1adc8be99c6a 100644 --- a/docs/how_to/tutorials/cross_compilation_and_rpc.py +++ b/docs/how_to/tutorials/cross_compilation_and_rpc.py @@ -84,7 +84,6 @@ # # INFO:root:RPCServer: bind to 0.0.0.0:9090 # - ###################################################################### # Declare and Cross Compile Kernel on Local Machine # ------------------------------------------------- @@ -201,6 +200,194 @@ cost = time_f(a, b).mean print(f"{cost:g} secs/op") +###################################################################### +# Scale RPC to Shared Devices +# --------------------------- +# +# The direct RPC server used above is the simplest way to run on one remote +# device. In shared environments, the same compile/upload/run flow is usually +# kept, but the connection is managed by an RPC tracker and, when needed, an +# RPC proxy. +# +# This setup is useful when: +# +# - multiple users or CI jobs share a small number of boards, +# - devices are registered by key rather than by fixed IP address, +# - the host cannot directly reach the device because of the network layout, or +# - the target device only has the minimal runtime stack needed for execution. +# +# The pieces fit together as follows: +# +# - **RPC server**: runs on the target device and executes uploaded modules. +# - **RPC tracker**: runs on a host and assigns matching RPC servers to clients. +# - **RPC proxy**: forwards traffic when the client cannot connect directly to +# the RPC server. +# +# .. figure:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/dev/how-to/rpc_system_suggested_arch.svg +# :align: center +# :width: 85% +# +# In the figure above, machine A connects through the tracker. Machine B runs +# an RPC proxy because machines C and D are not directly reachable from A. The +# tracker keeps a queue per RPC key. If a matching server is available, it is +# assigned to the client; otherwise, the request waits in that key's queue. +# +# Start the Tracker and Proxy +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# The tracker and proxy generally run on a host machine, not on the target +# device. They do not require target-specific drivers. +# +# .. code-block:: shell +# +# python3 -m tvm.exec.rpc_tracker --host RPC_TRACKER_IP --port 9190 --port-end 9191 +# +# .. code-block:: shell +# +# python3 -m tvm.exec.rpc_proxy \ +# --host RPC_PROXY_IP \ +# --port 9090 \ +# --port-end 9091 \ +# --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT +# +# Replace the host names, ports, and port ranges for your environment. The +# ``--port-end`` option is useful in CI because it prevents the service from +# silently choosing an unexpected port. +# +# Package a Minimal RPC Runtime +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# If the target can build TVM directly, install TVM on the target and launch +# the RPC server there. Otherwise, cross-compile the TVM runtime and package +# it with the Python RPC server. +# +# A typical CMake toolchain file for 64-bit ARM Linux looks like this: +# +# .. code-block:: cmake +# +# set(CMAKE_SYSTEM_NAME Linux) +# set(root_dir "/XXX/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu") +# +# set(CMAKE_C_COMPILER "${root_dir}/bin/aarch64-linux-gnu-gcc") +# set(CMAKE_CXX_COMPILER "${root_dir}/bin/aarch64-linux-gnu-g++") +# set(CMAKE_SYSROOT "${root_dir}/aarch64-linux-gnu/libc") +# +# set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) +# set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) +# set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) +# set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) +# +# Build the runtime from the TVM repository root. Enable target-specific +# options such as ``USE_OPENCL`` or vendor runtime support in ``config.cmake``. +# Build any target-specific runtime libraries that your deployment needs, such +# as ``tvm_runtime_opencl`` for OpenCL. +# +# .. code-block:: shell +# +# mkdir cross_build +# cd cross_build +# cp ../cmake/config.cmake ./ +# +# # Enable other options as needed, e.g. USE_OPENCL or vendor runtimes. +# sed -i "s|USE_LLVM.*)|USE_LLVM OFF)|" config.cmake +# +# cmake -DCMAKE_TOOLCHAIN_FILE=/YYY/aarch64-linux-gnu.cmake -DCMAKE_BUILD_TYPE=Release .. +# cmake --build . --target runtime -j +# # Optional example when USE_OPENCL is enabled: +# # cmake --build . --target tvm_runtime_opencl -j +# cd .. +# +# Then package the Python RPC server with the cross-compiled runtime and copy +# it to the device. +# +# .. code-block:: shell +# +# rm -rf tvm_runtime_package +# mkdir tvm_runtime_package +# cp -a python tvm_runtime_package/ +# cp cross_build/lib/libtvm_ffi.so tvm_runtime_package/python/tvm/ +# cp cross_build/lib/libtvm_runtime*.so tvm_runtime_package/python/tvm/ +# tar -czf tvm_runtime.tar.gz -C tvm_runtime_package python +# +# On the target device: +# +# .. code-block:: shell +# +# tar -xzf tvm_runtime.tar.gz +# export PYTHONPATH=`pwd`/python:${PYTHONPATH} +# +# Launch the Server +# ~~~~~~~~~~~~~~~~~ +# +# Launch the RPC server on the target device. Use the proxy form when the +# server connects through an RPC proxy; otherwise connect directly to the +# tracker. +# +# .. code-block:: shell +# +# # Through an RPC proxy. +# python3 -m tvm.exec.rpc_server \ +# --host RPC_PROXY_IP \ +# --port RPC_PROXY_PORT \ +# --through-proxy \ +# --key RPC_KEY +# +# # Directly to an RPC tracker. +# python3 -m tvm.exec.rpc_server \ +# --tracker RPC_TRACKER_IP:RPC_TRACKER_PORT \ +# --key RPC_KEY +# +# Query the tracker from the host to confirm that the servers are visible: +# +# .. code-block:: shell +# +# python3 -m tvm.exec.query_rpc_tracker --host RPC_TRACKER_IP --port RPC_TRACKER_PORT +# +# If three servers connect through a proxy, the output should look similar to: +# +# .. code-block:: text +# +# Tracker address RPC_TRACKER_IP:RPC_TRACKER_PORT +# +# Server List +# ---------------------------- +# server-address key +# ---------------------------- +# RPC_PROXY_IP:RPC_PROXY_PORT server:proxy[RPC_KEY0,RPC_KEY1,RPC_KEY2] +# ---------------------------- +# +# Queue Status +# --------------------------------------- +# key total free pending +# --------------------------------------- +# RPC_KEY0 0 0 3 +# --------------------------------------- +# +# Once the tracker assigns a server, the client-side code still follows the +# same pattern used earlier in this tutorial. Only the session creation +# changes from a direct connection to a tracker request: +# +# .. code-block:: python +# +# tracker = rpc.connect_tracker("RPC_TRACKER_IP", RPC_TRACKER_PORT) +# remote = tracker.request("RPC_KEY", priority=0, session_timeout=600) +# +# After that, use the same ``remote.upload()``, ``remote.load_module()``, remote +# device creation, and ``time_evaluator`` flow shown above. +# +# Troubleshooting Minimal Device Environments +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# Some target devices have intentionally small Python environments. The TVM +# runtime itself does not require full NumPy support for RPC execution, but the +# Python RPC server may import modules that import ``numpy``. If installing or +# cross-compiling NumPy is not practical, a small ``numpy.py`` shim in the +# device's Python ``site-packages`` directory can be enough for server startup. +# +# If ``cloudpickle`` is missing, copy it from another Python environment into +# the device's ``site-packages`` directory. It is a pure Python package, so it +# usually does not need cross-compilation. +# ######################################################################### # Run OpenCL Kernel Remotely by RPC # --------------------------------- diff --git a/docs/index.rst b/docs/index.rst index 2c66c4295d26..5f218d9e6fc8 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -49,7 +49,6 @@ driving its costs down. how_to/tutorials/export_and_load_executable how_to/tutorials/mix_python_and_tvm_with_pymodule how_to/tutorials/bring_your_own_codegen - how_to/dev/index .. The Deep Dive content is comprehensive .. we maintain a ``maxdepth`` of 2 to display more information on the main page. @@ -62,6 +61,12 @@ driving its costs down. deep_dive/tensor_ir/index deep_dive/relax/index +.. toctree:: + :maxdepth: 1 + :caption: Troubleshooting + + errors + .. toctree:: :maxdepth: 1 :caption: API Reference From f1ede3ad58596bb4078923dddc8e1c6123b4b61f Mon Sep 17 00:00:00 2001 From: tlopex <820958424@qq.com> Date: Tue, 26 May 2026 01:05:30 -0400 Subject: [PATCH 2/2] finish3 --- docs/errors.rst | 4 ++-- docs/index.rst | 7 +------ 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/docs/errors.rst b/docs/errors.rst index 3cc43461634e..be3c6df83fc2 100644 --- a/docs/errors.rst +++ b/docs/errors.rst @@ -16,8 +16,8 @@ under the License. -Troubleshooting TVM Errors -========================== +TVM Errors +========== TVM may raise errors from Python code, from C++ code reached through the FFI, or from generated runtime modules. Error messages usually include diff --git a/docs/index.rst b/docs/index.rst index 5f218d9e6fc8..87af6cc267c6 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -36,6 +36,7 @@ driving its costs down. install/index get_started/tutorials/quick_start get_started/tutorials/ir_module + errors .. toctree:: :maxdepth: 1 @@ -61,12 +62,6 @@ driving its costs down. deep_dive/tensor_ir/index deep_dive/relax/index -.. toctree:: - :maxdepth: 1 - :caption: Troubleshooting - - errors - .. toctree:: :maxdepth: 1 :caption: API Reference