dmd.root.array: add any and all predicates to dmd.root.array

[gorsing]

Description:
Adds high-performance, inlined any and all predicates to Array in dmd.root.array. These templates use alias predicates to ensure zero-overhead execution, facilitating the refactoring of manual for loops in semantic analysis stages.

[dlang-bot]

Thanks for your pull request and interest in making D better, @gorsing! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#22284"

[dkorpel]

Can you include some examples of where you want to use these functions?

[gorsing]

Hi @dkorpel, thanks for the question.
I've updated dmd/semantic3.d to use these new functions. You can see how they replace manual for and foreach loops with more concise any and all predicates.

[dkorpel]

Thanks for the examples. Unfortunately I don't think they demonstrate benefits. It would be helpful if it collapsed complex loop logic into something simpler, but replacing a foreach loop body with a function literal passed to a call to all or any only makes the code harder to read in my opinion, as it's now harder to recognize that a loop is being performed.

[dkorpel]

This example adds a return statement, which is discarded.

[dkorpel]

This adds more code, doing an additional loop which is redundant because the original loop is unchanged.

[dkorpel]

This is an improvement relative to for, but relative to the more obvious refactor to foreach it only adds another discarded return value.

[dkorpel]

This is a good loop replacement, but can already be done better with a.contains(f).

[gorsing]

Thanks for the feedback, @dkorpel.
​You're right, these specific spots in semantic3.d weren't the best choice for functional primitives, especially since they either introduced redundant loops or made the logic less readable compared to a standard foreach. I will revert the changes in this file.
​I also take your point about using a.contains(f) instead of a manual any call for simple membership checks.
​Out of curiosity, are there specific areas in the DMD codebase where you feel all or any are preferred? Is it mostly for collapsing deeply nested/complex loop logic, or is there a general preference for foreach even in those cases to keep the 'loop' nature of the code explicit?"

[dkorpel]

​Out of curiosity, are there specific areas in the DMD codebase where you feel all or any are preferred?

Not really. I can think of many places where a loop is used to find an element in a list. any can be used for that, but find or contains are a better fit.

Is it mostly for collapsing deeply nested/complex loop logic, or is there a general preference for foreach even in those cases to keep the 'loop' nature of the code explicit?"

If you can formulate a piece of loop code as a mathematical question (like the aforementioned "does this list contain this element?") then the loop could be seen as an implementation detail, justifying the use of range functions. In dmd loops often have side effects and do more than produce a boolean result (computing types, printing errors, rewriting elements), in which case trying to hide the foreach part behind a functional interface is only obfuscating in my opinion.

[gorsing]

Hi @dkorpel Thanks for the previous review

I have updated our base Array!T container with several high-performance methods O(N). These are the modern practices found in Rust, LLVM, and modern C++ that were missing from our implementation.
Why this is important:

Instant Deletion (swapRemove):

• Before: O(N) (Slow: shifting all elements after the index).
• After: O(1) (Instant: swaps the target with the last element).
• Industry Standard: Matches vec.swap_remove() in Rust.

Efficient Filtering (removeIf):

• Before: O(N2) (Very slow: repeated deletions cause nested shifts).
• After: O(N) (Fast: the "Erase-Remove" idiom, processes the whole array in a single pass).
• Industry Standard: Matches vec.retain() in Rust and std::erase_if in C++20.

Memory Management (shrinkToFit):

• The array can now release unused capacity back to the system or move data back to the fast SSO (Small Size Optimization) buffer.
• This is critical for preventing memory bloating in long-running processes.

Cleaner Code (any, all):

• Replaces bulky foreach loops with expressive, functional-style checks.
• Uses short-circuiting: the search stops the moment the result is known, saving CPU cycles.

[gorsing]

I have taken out some parts of MR as separate problems
#22300
#22301

[dkorpel]

I have taken out some parts of MR as separate problems

Thanks for doing that!

[dkorpel]

I have updated our base Array!T container with several high-performance methods O(N).

Do you have benchmarks showing how much this improves compilation times? Note that the dmd.root.array module is not a complete general purpose array implementation, it is only used for dmd to make bootstrapping easier. Hence additions to that module need to be motivated through dmd's use cases.

[gorsing]

Hi @dkorpel
I've highlighted the changes to the old code in #22327 to separate the addition of new methods from fixes or improvements.

[gorsing]

I have updated our base Array!T container with several high-performance methods O(N).

Do you have benchmarks showing how much this improves compilation times? Note that the dmd.root.array module is not a complete general purpose array implementation, it is only used for dmd to make bootstrapping easier. Hence additions to that module need to be motivated through dmd's use cases.

I am currently working on a profiler report to demonstrate the specific performance gains.

Regarding total compilation time: measuring a macro-level improvement right now is non-trivial. Since I extracted the refactoring of semantic3.d into a separate PR to keep this contribution minimal, these new methods are not yet being utilized in the compiler's hot paths. Consequently, any immediate gains are currently lost in the noise of a full build.

However, the report I am preparing compares the current O(N²) patterns often found in DMD (manual loops with remove) against the O(N) removeIf implementation. My goal is to show that while dmd.root.array is indeed specialized, adding these algorithmic primitives provides the necessary infrastructure to optimize.

I will post the benchmark results shortly.

[dkorpel]

Consequently, any immediate gains are currently lost in the noise of a full build.

It is indeed hard, but important to measure because expectations and reality can wildly differ when trying to optimize something.

I will post the benchmark results shortly.

Great! If you need any help benchmarking let me know.