Add Fortio-Envoy optimization guide#29
Open
vaibhavk2 wants to merge 4 commits intointel:mainfrom
Open
Conversation
Signed-off-by: Vaibhav Shankar <vaibhav.shankar@intel.com>
Signed-off-by: Vaibhav Shankar <vaibhav.shankar@intel.com>
Signed-off-by: Vaibhav Shankar <vaibhav.shankar@intel.com>
rsiyer-intel
reviewed
May 4, 2026
rsiyer-intel
reviewed
May 4, 2026
rsiyer-intel
reviewed
May 4, 2026
rsiyer-intel
reviewed
May 4, 2026
rsiyer-intel
reviewed
May 4, 2026
rsiyer-intel
reviewed
May 4, 2026
Signed-off-by: Vaibhav Shankar <vaibhav.shankar@intel.com>
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
| | High LLC-load-misses in `perf stat` | Cross-NUMA memory access | Pin to single NUMA node | | ||
| | GC overhead in perf traces (`gcDrain`, `trygetfull`) | High allocation rate with default GC | Build Fortio with GreenTea GC (Go 1.25.1); raise `GOGC` | | ||
| | Envoy CPU bottlenecked in TLS | mTLS handshake overhead | Enable TLS session resumption. Make TLS communicaiton/handshake Async | | ||
| | NIC softirq on same cores as Envoy | IRQ affinity not set or not part of of same NUMA | Separate NIC IRQ cores from Envoy worker cores or ensure cores + mem + NIC are on same NUMA | |
Author
There was a problem hiding this comment.
did not understand this comment
Collaborator
There was a problem hiding this comment.
Suggested change
| | NIC softirq on same cores as Envoy | IRQ affinity not set or not part of of same NUMA | Separate NIC IRQ cores from Envoy worker cores or ensure cores + mem + NIC are on same NUMA | | |
| | NIC softirq on same cores as Envoy | IRQ affinity not set or not part of same NUMA | Separate NIC IRQ cores from Envoy worker cores or ensure cores + mem + NIC are on same NUMA | |
rsiyer-intel
reviewed
May 5, 2026
rsiyer-intel
reviewed
May 5, 2026
|
|
||
| --- | ||
| ### 5. Use fewer CPU cores as possible to run Envoy + fortio | ||
| Fortio and Envoy are intentionally run on a reduced number of CPU cores to avoid excessive spin-lock contention and busy-waiting behavior. By limiting core availability, the benchmark prevents threads from continuously spinning on shared locks and instead exposes meaningful contention, scheduling, and proxy-path behavior under realistic CPU pressure |
Collaborator
There was a problem hiding this comment.
Is the suggestion to increase docker cpu quota but restrict to fewer CPU cores? This is a bit confusing. Can you clarify/elaborate?
should cpu quota match the number of cores it is being pinned to
i.e. --cpus = -cpuset-cpus ""
rsiyer-intel
reviewed
May 5, 2026
|
|
||
| #### Worker and socket tuning | ||
| - **`SO_REUSEPORT`**: Already used by Envoy by default. Verify it is not disabled by any sysctls - it distributes `accept()` load evenly across worker threads without a shared accept mutex. | ||
| - **`--concurrency`**: Tune thread concurrency to match the number of physical cores assigned to the container. Over-provisioning causes false sharing, under-provisioning wastes hardware. |
Collaborator
There was a problem hiding this comment.
Same as bullet 2? --concurrency equal to --cpus
rsiyer-intel
reviewed
May 5, 2026
| ### 6. Other Envoy Tuning -- which can be explored | ||
|
|
||
| #### Worker and socket tuning | ||
| - **`SO_REUSEPORT`**: Already used by Envoy by default. Verify it is not disabled by any sysctls - it distributes `accept()` load evenly across worker threads without a shared accept mutex. |
Collaborator
There was a problem hiding this comment.
Will be useful to provide which sysctl setting to verify
rsiyer-intel
reviewed
May 6, 2026
|
|
||
| ### 2. Increase CPU Quota for Both Containers | ||
|
|
||
| With NUMA pinning in place, increasing the CPU quota for both Fortio and Envoy containers further reduces scheduling delays and spin lock overhead. Keep `--concurrency` equal to `--cpus` to avoid over-subscription. The numbers are just examples. Tune this based on the SKU/cores used. |
Collaborator
There was a problem hiding this comment.
On 256 threads system, you have allocated only 32+16 cores? What would be a good general recommendation? If someone had to tune based on SKU/cores they have?
rsiyer-intel
requested changes
May 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Fortio-Envoy optimization guide and related documentation.