tune default cpu properties#4323
Open
dtrawins wants to merge 4 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR tunes OpenVINO CPU default property selection in Docker to better handle throughput mode detection and multi-socket hosts, and adds targeted unit tests for the new decision logic.
Changes:
- Add CPU socket-count detection and refine physical-core-per-socket detection from
/proc/cpuinfo. - Refactor
applyDefaultCpuPropertiesinto a testable overload that capsnum_streamsby physical cores across sockets and avoids settinginference_num_threadswhen the environment appears unconstrained (e.g., HT-visible cores > physical cores). - Move CPU default property application from
ModelManagerinitialization to per-model compilation forCPU, and add unit tests covering constrained/unconstrained + latency/throughput cases.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/test/ov_utils_test.cpp |
Adds unit tests for the new applyDefaultCpuProperties overload and its Docker/throughput/latency behaviors. |
src/systeminfo.hpp |
Declares new getSocketsCount() API to support multi-socket-aware defaults. |
src/systeminfo.cpp |
Implements getSocketsCount() and adjusts physical core detection to be per-socket. |
src/ov_utils.hpp |
Exposes a testable overload of applyDefaultCpuProperties accepting explicit system parameters. |
src/ov_utils.cpp |
Implements new CPU default property logic (streams/threads/pinning) and delegates the runtime overload to system info helpers. |
src/modelmanager.cpp |
Removes global CPU property initialization on ov::Core construction. |
src/modelinstance.cpp |
Applies CPU defaults during CPU model compilation (instead of globally at core init). |
Comment on lines
+160
to
+164
| bool isThroughput = false; | ||
| const auto perfIt = properties.find(ov::hint::performance_mode.name()); | ||
| if (perfIt != properties.end()) { | ||
| try { | ||
| isThroughput = (perfIt->second.as<ov::hint::PerformanceMode>() == ov::hint::PerformanceMode::THROUGHPUT); |
Comment on lines
+244
to
+246
| #ifdef __linux__ | ||
| #include "../systeminfo.hpp" | ||
|
|
Comment on lines
+1026
to
+1033
| if (config.getTargetDevice() == "CPU") { | ||
| Status status = applyDefaultCpuProperties(pluginConfig); | ||
| if (!status.ok()) { | ||
| SPDLOG_LOGGER_ERROR(modelmanager_logger, "Failed to apply default CPU properties for model: {}; version: {}; error: {}", | ||
| getName(), getVersion(), status.string()); | ||
| return status; | ||
| } | ||
| } |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-188024
Fix detection of throughput mode and correct default parameters for multi socket host
🧪 Checklist
``