Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions Tutorials/ML/README-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ are filled from the options. Output types are left to ONNX Runtime.

Build O2Physics normally after loading its O2 environment. This target links
`O2::ML`, `O2::GPUCommon`, `ONNXRuntime::ONNXRuntime`, and the platform dynamic
loader. No HIP or CUDA headers are required for building the task. The O2 installation
must contain `ML/3rdparty/GPUORTFloat16.h` and its GPUCommon headers.
loader. No HIP or CUDA headers are required for building the task.

For AMD GPU runs, ONNX Runtime must contain **MIGraphXExecutionProvider**, and its
MIGraphX/ROCm shared libraries, including `libamdhip64.so`, must be discoverable
Expand Down Expand Up @@ -78,7 +77,7 @@ masking on AMD or `CUDA_VISIBLE_DEVICES` masking on NVIDIA. Run one benchmark pr
GPU. Repeat with the same batch sizes and CPU thread count, and compare several
runs. Small batches can be dominated by host overhead and transfers.

`--fp16 true` selects O2's `o2::OrtDataType::Float16_t` input storage and checks
`--fp16 true` selects O2's GCC/Clang's _Float16 input storage and checks
that the model input is FLOAT16. It does not convert model weights or prove that
every internal operation is FP16. Export/inspect the model accordingly. FP16
models retaining FP32 input interfaces need an adapted input path; this small
Expand Down
13 changes: 6 additions & 7 deletions Tutorials/ML/benchmarkOnnxModel.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
#include <Framework/ProcessingContext.h>
#include <Framework/runDataProcessing.h>

#include <ML/3rdparty/GPUORTFloat16.h>
#include <onnxruntime_c_api.h>
#include <onnxruntime_cxx_api.h>

Expand Down Expand Up @@ -196,15 +195,15 @@ struct BenchmarkOnnxModel {
// File loading and FP16 conversion are outside the timing loop. Without a
// file, retain deterministic synthetic inputs for performance-only runs.
std::vector<float> floats(fp16.value ? 0 : count);
std::vector<o2::OrtDataType::Float16_t> halves(fp16.value ? count : 0);
static_assert(sizeof(o2::OrtDataType::Float16_t) == 2);
std::vector<_Float16> halves(fp16.value ? count : 0);
static_assert(sizeof(_Float16) == 2);
auto store = [&](size_t i, float value) {
if (!std::isfinite(value)) {
throw std::runtime_error("Input features must be finite");
}
if (fp16.value) {
halves[i] = o2::OrtDataType::Float16_t(value);
if (!std::isfinite(halves[i].ToFloat())) {
halves[i] = _Float16(value);
if (!std::isfinite((float)halves[i])) {
throw std::runtime_error("Input feature overflows FP16");
}
} else {
Expand Down Expand Up @@ -295,11 +294,11 @@ struct BenchmarkOnnxModel {
throw std::runtime_error("Prediction export requires an FP32 or FP16 first output [batch-size, classes]");
}
const float* fp32 = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT ? output.GetTensorData<float>() : nullptr;
const auto* fp16Data = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16 ? output.GetTensorData<o2::OrtDataType::Float16_t>() : nullptr;
const auto* fp16Data = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16 ? output.GetTensorData<_Float16>() : nullptr;
const size_t columns = static_cast<size_t>(outputShape[1]);
// Validate before opening the destination, so invalid predictions do not
// replace an existing file with a partial matrix.
auto valueAt = [&](size_t i) { return fp32 ? fp32[i] : fp16Data[i].ToFloat(); };
auto valueAt = [&](size_t i) { return fp32 ? fp32[i] : (float)fp16Data[i]; };
for (size_t i = 0; i < info.GetElementCount(); ++i) {
if (!std::isfinite(valueAt(i))) {
throw std::runtime_error("Prediction export found a nonfinite value");
Expand Down
Loading