Hardware Prefetch Tuning for E-Cores v2 by IntelSwede · Pull Request #43 · intel/optimization-zone

IntelSwede · 2026-07-17T12:02:01Z

Fixed image, typos etc. based on review feedback.
Pointer from main readme to be added separately as additional HW material will be pushed.

adgubrud

I noticed there was a previous PR that you closed after initial comments. Please keep 1 PR open until all issues are resolved.

adgubrud · 2026-07-17T19:08:29Z

As this content is specific to E-core prefetchers, directory should be renamed, e.g., ecorePrefetchTuning.

adgubrud · 2026-07-17T19:08:45Z

Link to this document should be added to root README.md TOC.

adgubrud · 2026-07-17T19:10:03Z

+
+# 1 INTRODUCTION
+
+Hardware prefetchers are an excellent way of improving performance by fetching information ahead of time. The most basic prefetchers that only fetched the next cache line were introduced in the early days of processors. Today, prefetchers can track complex patterns to get the most relevant data into the caches ahead of usage. However, prefetching in multicore systems is a complex task; it can improve performance significantly, but driving more data into the caches can also evict more relevant data. Furthermore, prefetchers drive up the DDR bandwidth, which can become a bottleneck and lower overall system performance. By tuning the prefetchers, one can ensure optimal performance at any given time.


Suggested change

Hardware prefetchers are an excellent way of improving performance by fetching information ahead of time. The most basic prefetchers that only fetched the next cache line were introduced in the early days of processors. Today, prefetchers can track complex patterns to get the most relevant data into the caches ahead of usage. However, prefetching in multicore systems is a complex task; it can improve performance significantly, but driving more data into the caches can also evict more relevant data. Furthermore, prefetchers drive up the DDR bandwidth, which can become a bottleneck and lower overall system performance. By tuning the prefetchers, one can ensure optimal performance at any given time.

Hardware prefetchers are an excellent way of improving performance by fetching information ahead of time. The most basic prefetchers that only fetch the next cache line were introduced in the early days of processors. Today, prefetchers can track complex patterns to get the most relevant data into the caches ahead of usage. However, prefetching in multicore systems is a complex task. It can improve performance significantly, but driving more data into the caches can also evict more relevant data. Furthermore, prefetchers drive up the DDR bandwidth, which can become a bottleneck and lower overall system performance. By tuning the prefetchers, one can ensure optimal performance at any given time.

adgubrud · 2026-07-17T19:11:44Z

+
+The NLP is commonly referred to as the DCU Stream Prefetcher.
+
+The NLP prefetches the next line of any given access, applicable to data. The same functionality is also built into the front-end for instruction fetches. The NLP is one of the earliest types of prefetchers introduced at a time well before multi-core was the norm. It was a blunt approach: simple to implement, worked well for single-core systems, but drove much bandwidth. This impacted both DDR bandwidth and L1/L2 caches and queues. Use of the NLP should be done with care, and a general recommendation is to run with the NLP disabled unless you can observe a benefit to your system.


Suggested change

The NLP prefetches the next line of any given access, applicable to data. The same functionality is also built into the front-end for instruction fetches. The NLP is one of the earliest types of prefetchers introduced at a time well before multi-core was the norm. It was a blunt approach: simple to implement, worked well for single-core systems, but drove much bandwidth. This impacted both DDR bandwidth and L1/L2 caches and queues. Use of the NLP should be done with care, and a general recommendation is to run with the NLP disabled unless you can observe a benefit to your system.

The NLP prefetches the next line of any given access, applicable to data. The same functionality is also built into the front-end for instruction fetches. The NLP is one of the earliest types of prefetchers introduced at a time well before multi-core was the norm. It was a blunt approach: simple to implement, worked well for single-core systems, but drove a large amount of bandwidth. This impacted both DDR bandwidth and L1/L2 caches and queues. Use of the NLP should be done with care, and a general recommendation is to run with the NLP disabled unless you can observe a benefit to your system.

adgubrud · 2026-07-17T19:12:24Z

+
+The type of requests used to train and trigger the L2 prefetcher is one of the configurable parameters; see Section [4.5](#45-msr-0x1323). The L2 prefetchers are highly configurable. However, additional mechanisms not covered in this paper ensure fairness, overload protection, and timely relevance in the requests.
+
+A central feedback parameter is the depth of the XQ. The XQ threshold level for when to issue prefetches is also a valuable configuration parameter present in multiple MSRs. A higher number of entries in that queue states that more loads go beyond the L2 cache; hence, the L3 cache will see a higher load. If these miss in the L3 cache, they will likely generate a significant DDR load, for which it is generally a good practice to reduce the number of prefetches issued. Requests towards MMIO address space (such as PCIe) will also be routed through the XQ but are seldom a significant load.


Suggested change

A central feedback parameter is the depth of the XQ. The XQ threshold level for when to issue prefetches is also a valuable configuration parameter present in multiple MSRs. A higher number of entries in that queue states that more loads go beyond the L2 cache; hence, the L3 cache will see a higher load. If these miss in the L3 cache, they will likely generate a significant DDR load, for which it is generally a good practice to reduce the number of prefetches issued. Requests towards MMIO address space (such as PCIe) will also be routed through the XQ but are seldom a significant load.

A central feedback parameter is the depth of the XQ. The XQ threshold level for when to issue prefetches is also a valuable configuration parameter present in multiple MSRs. A higher number of entries in that queue states that more loads go beyond the L2 cache. Hence, the L3 cache will see a higher load. If these miss in the L3 cache, they will likely generate a significant DDR load, for which it is generally a good practice to reduce the number of prefetches issued. Requests towards MMIO address space (such as PCIe) will also be routed through the XQ but are seldom a significant load.

adgubrud · 2026-07-17T19:21:55Z

+
+This event measures the number of load operations concluded where the load hit in local or remote memory. This can be leveraged to identify how much of the overall memory bandwidth originates from the respective core. A more fine-grained selection of which cores should be throttled can thus be made.
+
+See also event MEM_BOUND_STALLS_LOAD.LLC_MISS_LOCAL_MEM which counts the cycle cost for the penalty of going to local memory. Additionally, MEM_BOUND_STALLS_LOAD.LLC_MISS_REMOTEMEM/OTHERMOD will include access to remote memory and another modules cache. Beginning on DKT, this event counts only at retirement.


I think it makes sense to provide the full form of Darkmont here

Suggested change

See also event MEM_BOUND_STALLS_LOAD.LLC_MISS_LOCAL_MEM which counts the cycle cost for the penalty of going to local memory. Additionally, MEM_BOUND_STALLS_LOAD.LLC_MISS_REMOTEMEM/OTHERMOD will include access to remote memory and another modules cache. Beginning on DKT, this event counts only at retirement.

See also event MEM_BOUND_STALLS_LOAD.LLC_MISS_LOCAL_MEM which counts the cycle cost for the penalty of going to local memory. Additionally, MEM_BOUND_STALLS_LOAD.LLC_MISS_REMOTEMEM/OTHERMOD will include access to remote memory and another modules cache. Beginning on Darkmont, this event counts only at retirement.

adgubrud · 2026-07-17T19:22:20Z

+
+This event counts the number of cacheable memory requests that miss in the LLC and hence targets DDR or other coherent memory.
+
+Alternatively, event OCR.READS_TO_CORE.L3_MISS also counts cacheable reads including demand and prefetched data reads, RFOs, and code reads.


Suggested change

Alternatively, event OCR.READS_TO_CORE.L3_MISS also counts cacheable reads including demand and prefetched data reads, RFOs, and code reads.

Alternatively, event OCR.READS_TO_CORE.L3_MISS also counts cacheable reads including demand and prefetched data reads, read for ownership (RFO), and code reads.

adgubrud · 2026-07-17T19:22:43Z

+
+#### OCR.hwpf\*
+
+The Off-Core Response events which monitor the traffic, requests and responses, to and from the uncore, provide a breakdown for the L1, L2, and L3 prefetchers. These events can provide a ratio of prefetches per cache level versus demands for those that miss the L2 cache. OCR.HWPF.ANY_RESPONSE will count all instances for all 3 levels. The breakdown then includes: OCR.HWPF_L1D.ANY_RESPONSE, OCR.HWPF_L2.ANY_RESPONSE, OCR.HWPF_L3.ANY_RESPONSE.


Suggested change

The Off-Core Response events which monitor the traffic, requests and responses, to and from the uncore, provide a breakdown for the L1, L2, and L3 prefetchers. These events can provide a ratio of prefetches per cache level versus demands for those that miss the L2 cache. OCR.HWPF.ANY_RESPONSE will count all instances for all 3 levels. The breakdown then includes: OCR.HWPF_L1D.ANY_RESPONSE, OCR.HWPF_L2.ANY_RESPONSE, OCR.HWPF_L3.ANY_RESPONSE.

The Off-Core Response events which monitor the request and response traffic to and from the uncore and provide a breakdown for the L1, L2, and L3 prefetchers. These events can provide a ratio of prefetches per cache level versus demands for those that miss the L2 cache. OCR.HWPF.ANY_RESPONSE will count all instances for all 3 levels. The breakdown then includes: OCR.HWPF_L1D.ANY_RESPONSE, OCR.HWPF_L2.ANY_RESPONSE, OCR.HWPF_L3.ANY_RESPONSE.

adgubrud · 2026-07-17T19:26:32Z

+```
+
+## 4.3 MSR 0x1321
+


If possible, please provide what effect providing specific values has. For instance for MSR 0x1321 0:0, does setting it to 1 enable tracking of instruction flows? Do you have to set to 1?

adgubrud · 2026-07-17T19:26:47Z

+|   CMT   |   4   | 19:16 | Sets the number of new AMP prefetches added to the number of pending prefetches for each successful training                                         |
+|   CMT   |   1   | 20:20 | Set to 1 to turn on the demand based L2 prefetch throttler                                                                                               |
+|   GRT   |   8   | 28:21 | MLC Streamer demand density.                                                                                                                             |
+|   GRT   |   4   | 32:29 | MLC Streamer demand density override.                                                                                                                    |


What does density override do?

IntelSwede added 3 commits July 2, 2026 16:53

Hardware Prefetch Tuning for E-Cores

8df4c38

Uppdated based on feedback

46c6836

Fixed figure caption

edcd25f

adgubrud requested changes Jul 17, 2026

View reviewed changes

adgubrud requested a review from jasukhar July 17, 2026 19:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Hardware Prefetch Tuning for E-Cores v2#43

Hardware Prefetch Tuning for E-Cores v2#43
IntelSwede wants to merge 3 commits into
intel:mainfrom
IntelSwede:hwpf

IntelSwede commented Jul 17, 2026

Uh oh!

adgubrud left a comment

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

adgubrud Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		# 1 INTRODUCTION

		Hardware prefetchers are an excellent way of improving performance by fetching information ahead of time. The most basic prefetchers that only fetched the next cache line were introduced in the early days of processors. Today, prefetchers can track complex patterns to get the most relevant data into the caches ahead of usage. However, prefetching in multicore systems is a complex task; it can improve performance significantly, but driving more data into the caches can also evict more relevant data. Furthermore, prefetchers drive up the DDR bandwidth, which can become a bottleneck and lower overall system performance. By tuning the prefetchers, one can ensure optimal performance at any given time.


		The NLP is commonly referred to as the DCU Stream Prefetcher.

		The NLP prefetches the next line of any given access, applicable to data. The same functionality is also built into the front-end for instruction fetches. The NLP is one of the earliest types of prefetchers introduced at a time well before multi-core was the norm. It was a blunt approach: simple to implement, worked well for single-core systems, but drove much bandwidth. This impacted both DDR bandwidth and L1/L2 caches and queues. Use of the NLP should be done with care, and a general recommendation is to run with the NLP disabled unless you can observe a benefit to your system.


		The type of requests used to train and trigger the L2 prefetcher is one of the configurable parameters; see Section [4.5](#45-msr-0x1323). The L2 prefetchers are highly configurable. However, additional mechanisms not covered in this paper ensure fairness, overload protection, and timely relevance in the requests.

		A central feedback parameter is the depth of the XQ. The XQ threshold level for when to issue prefetches is also a valuable configuration parameter present in multiple MSRs. A higher number of entries in that queue states that more loads go beyond the L2 cache; hence, the L3 cache will see a higher load. If these miss in the L3 cache, they will likely generate a significant DDR load, for which it is generally a good practice to reduce the number of prefetches issued. Requests towards MMIO address space (such as PCIe) will also be routed through the XQ but are seldom a significant load.


		This event measures the number of load operations concluded where the load hit in local or remote memory. This can be leveraged to identify how much of the overall memory bandwidth originates from the respective core. A more fine-grained selection of which cores should be throttled can thus be made.

		See also event MEM_BOUND_STALLS_LOAD.LLC_MISS_LOCAL_MEM which counts the cycle cost for the penalty of going to local memory. Additionally, MEM_BOUND_STALLS_LOAD.LLC_MISS_REMOTEMEM/OTHERMOD will include access to remote memory and another modules cache. Beginning on DKT, this event counts only at retirement.


		This event counts the number of cacheable memory requests that miss in the LLC and hence targets DDR or other coherent memory.

		Alternatively, event OCR.READS_TO_CORE.L3_MISS also counts cacheable reads including demand and prefetched data reads, RFOs, and code reads.


		#### OCR.hwpf\*

		The Off-Core Response events which monitor the traffic, requests and responses, to and from the uncore, provide a breakdown for the L1, L2, and L3 prefetchers. These events can provide a ratio of prefetches per cache level versus demands for those that miss the L2 cache. OCR.HWPF.ANY_RESPONSE will count all instances for all 3 levels. The breakdown then includes: OCR.HWPF_L1D.ANY_RESPONSE, OCR.HWPF_L2.ANY_RESPONSE, OCR.HWPF_L3.ANY_RESPONSE.

Uh oh!

Conversation

IntelSwede commented Jul 17, 2026

Uh oh!

adgubrud left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants