Skip to content

Commit fc7d9f0

Browse files
committed
ONNX: Use GCC/clang float16 instead of ML/3rdparty/GPUORTFloat16.h
1 parent be13808 commit fc7d9f0

2 files changed

Lines changed: 8 additions & 10 deletions

File tree

‎Tutorials/ML/README-benchmark.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,7 @@ are filled from the options. Output types are left to ONNX Runtime.
1010

1111
Build O2Physics normally after loading its O2 environment. This target links
1212
`O2::ML`, `O2::GPUCommon`, `ONNXRuntime::ONNXRuntime`, and the platform dynamic
13-
loader. No HIP or CUDA headers are required for building the task. The O2 installation
14-
must contain `ML/3rdparty/GPUORTFloat16.h` and its GPUCommon headers.
13+
loader. No HIP or CUDA headers are required for building the task.
1514

1615
For AMD GPU runs, ONNX Runtime must contain **MIGraphXExecutionProvider**, and its
1716
MIGraphX/ROCm shared libraries, including `libamdhip64.so`, must be discoverable
@@ -78,7 +77,7 @@ masking on AMD or `CUDA_VISIBLE_DEVICES` masking on NVIDIA. Run one benchmark pr
7877
GPU. Repeat with the same batch sizes and CPU thread count, and compare several
7978
runs. Small batches can be dominated by host overhead and transfers.
8079

81-
`--fp16 true` selects O2's `o2::OrtDataType::Float16_t` input storage and checks
80+
`--fp16 true` selects O2's GCC/Clang's _Float16 input storage and checks
8281
that the model input is FLOAT16. It does not convert model weights or prove that
8382
every internal operation is FP16. Export/inspect the model accordingly. FP16
8483
models retaining FP32 input interfaces need an adapted input path; this small

‎Tutorials/ML/benchmarkOnnxModel.cxx‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
#include <Framework/ProcessingContext.h>
1717
#include <Framework/runDataProcessing.h>
1818

19-
#include <ML/3rdparty/GPUORTFloat16.h>
2019
#include <onnxruntime_c_api.h>
2120
#include <onnxruntime_cxx_api.h>
2221

@@ -196,15 +195,15 @@ struct BenchmarkOnnxModel {
196195
// File loading and FP16 conversion are outside the timing loop. Without a
197196
// file, retain deterministic synthetic inputs for performance-only runs.
198197
std::vector<float> floats(fp16.value ? 0 : count);
199-
std::vector<o2::OrtDataType::Float16_t> halves(fp16.value ? count : 0);
200-
static_assert(sizeof(o2::OrtDataType::Float16_t) == 2);
198+
std::vector<_Float16> halves(fp16.value ? count : 0);
199+
static_assert(sizeof(_Float16) == 2);
201200
auto store = [&](size_t i, float value) {
202201
if (!std::isfinite(value)) {
203202
throw std::runtime_error("Input features must be finite");
204203
}
205204
if (fp16.value) {
206-
halves[i] = o2::OrtDataType::Float16_t(value);
207-
if (!std::isfinite(halves[i].ToFloat())) {
205+
halves[i] = _Float16(value);
206+
if (!std::isfinite((float)halves[i])) {
208207
throw std::runtime_error("Input feature overflows FP16");
209208
}
210209
} else {
@@ -295,11 +294,11 @@ struct BenchmarkOnnxModel {
295294
throw std::runtime_error("Prediction export requires an FP32 or FP16 first output [batch-size, classes]");
296295
}
297296
const float* fp32 = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT ? output.GetTensorData<float>() : nullptr;
298-
const auto* fp16Data = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16 ? output.GetTensorData<o2::OrtDataType::Float16_t>() : nullptr;
297+
const auto* fp16Data = outputType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16 ? output.GetTensorData<_Float16>() : nullptr;
299298
const size_t columns = static_cast<size_t>(outputShape[1]);
300299
// Validate before opening the destination, so invalid predictions do not
301300
// replace an existing file with a partial matrix.
302-
auto valueAt = [&](size_t i) { return fp32 ? fp32[i] : fp16Data[i].ToFloat(); };
301+
auto valueAt = [&](size_t i) { return fp32 ? fp32[i] : (float)fp16Data[i]; };
303302
for (size_t i = 0; i < info.GetElementCount(); ++i) {
304303
if (!std::isfinite(valueAt(i))) {
305304
throw std::runtime_error("Prediction export found a nonfinite value");

0 commit comments

Comments
 (0)