Fix poisoning leftover for vector and string ASan annotations by amyw-msft · Pull Request #6285 · microsoft/STL

amyw-msft · 2026-05-13T23:35:30Z

AddressSanitizer poisons memory in 8 byte chunks. It can mark a memory region as 'partially addressable', but only by marking the first x bytes as valid to access and remaining as poisoned. There is no way to say "only the first x bytes are poisoned". The result is that the ends of buffers that are not 8-byte aligned either need to over-poison past the end of the buffer, or under-poison and allow potential buffer overflow to go unnoticed under ASan.

For ASan container annotations, we choose which strategy to use based on the allocator. If we can be sure that the allocator only allocations in 8-byte aligned chunks, then we can safely over-poison past the end of the buffer. However, when we do this (increase the pointer that points to the 'end'), we also need to adjust the actual 'last valid' pointer as well if they are equal, because the way we clean up the container poisoning is by calling that annotations API, saying that the last valid and the end pointer are the same.

This change checks for this condition and advances the 'last valid' pointer if it is equal to the 'end' pointer. Without this fix, shadow bytes will remain poisoned after an annotated container leaves scope.

…'end' point in situations where the end has been extended to the end of the allocation block.

Copilot

Pull request overview

Fixes ASan container annotation cleanup for std::vector and std::basic_string when the allocator is at least 8-byte aligned. In that path, the implementation over-poisons up to the next 8-byte shadow boundary past _End. When removing or shrinking the annotation, the "old/new last valid" pointers passed to __sanitizer_annotate_contiguous_container must also be bumped to that same aligned boundary whenever they equal _End; otherwise the trailing shadow bytes past _End remain poisoned after the container goes out of scope.

Changes:

In stl/inc/vector's aligned-allocator branch of _Apply_annotation, advance _Old_last/_New_last to the aligned-after-end pointer when they equal _End.
Apply the analogous fix to stl/inc/xstring's _Apply_annotation, and restructure the function so the aligned branch returns early.
Add a new regression test (GH_006276_annotation_poison_cleanup) exercising both the under-poison and over-poison paths for string and vector with an ASan-unaware arena allocator.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
stl/inc/vector	Compute aligned old/new last pointers and pass them to the sanitizer call in the aligned-allocator branch.
stl/inc/xstring	Same fix for `basic_string`; restructure so the aligned branch returns early before the non-aligned code.
tests/std/tests/GH_006276_annotation_poison_cleanup/test.cpp	New regression test verifying shadow bytes are fully cleared after container destruction.
tests/std/tests/GH_006276_annotation_poison_cleanup/env.lst	ASan test matrix for the new test.
tests/std/test.lst	Register the new test directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

davidmrdavid

LGTM, but will defer to STL. Just a small observation

StephanTLavavej · 2026-05-14T19:34:45Z

ARM64 crash in the new test; is this something concerning, or pre-existing rare flakiness?

Test step failed unexpectedly.
Command: "C:\stlBuild\tests\std\tests\GH_006276_annotation_poison_cleanup\Output\15\GH_006276_annotation_poison_cleanup.exe"
Exit Code: 3221225477 (0xC0000005)

zacklj89

I don't immediately know what could be causing the ARM64 failure, but let me know if I can help investigate.

zacklj89 · 2026-05-18T16:08:47Z

            const void* const _Old_last = _STD _Unfancy(_Old_last_);
            const void* const _New_last = _STD _Unfancy(_New_last_);
            if constexpr ((_Container_allocation_minimum_asan_alignment<vector>) >= _Asan_granularity) {
+                // If we realign the _End forward to maximize coverage, we need to keep other significant points


By "significant points" here, we mean _Old_last and _New_last right? Maybe we could clarify if there's another push:

// If we realign _End forward to maximize coverage, also realign _Old_last and _New_last when they // equal _End. Otherwise, the ASan annotation may leave stale shadow bytes behind when cleaning up.

zacklj89 · 2026-05-18T16:12:11Z

+
+                // Allocation is potentially unaligned, so we cannot annotate whole buffer since we might be
+                // entering memory owned by someone else. Therefore, pull back where the annotations end on the buffer,
+                // but may miss some coverage near end of buffer.


Another comment comment, but feel free to not integrate it. I haven't worked as closely here as you, so maybe this is redundant, but:

// The allocation may not end on an ASan granularity boundary. In that case, annotating up to _End // could affect shadow bytes for memory beyond this allocation, so restrict annotations to the largest // aligned subrange inside the buffer. This may leave the tail of the buffer unannotated.

zacklj89 · 2026-05-18T16:15:11Z

+
+                const void* const _New_end_aligned  = _STD _Get_asan_aligned_after(_End);
+                const void* const _Old_last_aligned = (_Old_last == _End) ? _New_end_aligned : _Old_last;
+                const void* const _New_last_aligned = (_New_last == _End) ? _New_end_aligned : _New_last;


Nice work 🥲

zacklj89 · 2026-05-18T16:20:44Z

+    test_arena.print_shadow();
+}
+
+int main() {


Do you think it would be worthwhile to add a test here that does like shrink from full and tests the shadow, like testing _Old_last == _End and doing pop_back() on a full vector/string?

Found more issues testing this. Latest change addresses the issues. Also decided to integrate the kind of testing I'm doing here into the larger string and vector annotation tests to ensure full coverage.

… imbue a sense of intent at the call site (i.e. 'end' pointers should be realigned when overpoisoning, but all other pointers should not - other attempts to capture this were convoluted). Moved testing into pre-existing vector/string annotation tests, with an added allocator that will be able to detect 'left-poison-bytes-behind' issues as well as a microsoftgh-6276 specific test case and extra testing for finding similar cases (insert/remove triggering a realloc).

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.

amyw-msft · 2026-06-05T22:02:34Z

+    _CONSTEXPR20 const void* _Annotation_begin() const noexcept {
+        _STL_INTERNAL_CHECK(_Mypair._Myval2._Myfirst != nullptr);
+        return _STD _Unfancy(_Mypair._Myval2._Myfirst);
    }

-    _CONSTEXPR20 void _Modify_annotation(const difference_type _Count) const noexcept {
-        // Extends/shrinks the annotated range by _Count
-        if (_Count == 0) { // nothing to do
-            // This also avoids calling _Apply_annotation() with null pointers
-            // when the vector has zero capacity, see GH-2464.
-            return;
+    _CONSTEXPR20 const void* _Annotation_at(const size_type _Index) const noexcept {
+        _STL_INTERNAL_CHECK(_Mypair._Myval2._Myfirst != nullptr);
+        return _STD _Unfancy(_Mypair._Myval2._Myfirst + _Index);
+    }


I'll edit the comments in xmemory and remove any unused code.

The pointers passed in to __sanitizer_annotate_contiguous_container are permitted to be unaligned w.r.t. shadow granularity.

If the beginning is not aligned, ASan still handles the first shadow byte correctly and doesn't need extra over-poisoning logic like the end, because unlike at the end of the buffer (which would require "first X bytes are poisoned" semantics), we do have the semantics to say "first X bytes are valid".

If the end is not aligned, ASan rounds down to the nearest shadow granularity. For this reason, if the STL knows no one will poison the rest of the final shadow byte due to the allocator alignment, the STL will round up before passing the pointer (our 'over-poisoning' technique) to ensure no coverage is lost at the end of the buffer.

+        if constexpr ((_Container_allocation_minimum_asan_alignment<vector>) >= _Asan_granularity) {
+            // If the allocator alignment is less than or equal to the ASan granularity, then we can
+            // realign the end forward to maximize the annotation coverage.
+            return _STD _Get_asan_aligned_after(_End);
+        } else {
+            // The allocation may not end on an ASan granularity boundary. In that case, annotating up to _End
+            // could affect shadow bytes for memory beyond this allocation, so restrict annotations to the largest
+            // aligned subrange inside the buffer. This may leave the tail of the buffer unannotated.
+
+            return _End; // Using unaligned end will snap it to previous ASan granularity boundary.


+    _CONSTEXPR20 const void* _Annotation_begin() const noexcept {
+        return _Mypair._Myval2._Myptr();
    }


+        constexpr bool _Large_string_always_asan_aligned =
+            (_Container_allocation_minimum_asan_alignment<basic_string>) >= _Asan_granularity;
+
+        if constexpr (_Large_string_always_asan_aligned) {
+            // If the allocator alignment is less than or equal to the ASan granularity, then we can
+            // realign the end forward to maximize the annotation coverage.
+            return _STD _Get_asan_aligned_after(_End);
+        } else {
+            // The allocation may not end on an ASan granularity boundary. In that case, annotating up to _End
+            // could affect shadow bytes for memory beyond this allocation, so restrict annotations to the largest
+            // aligned subrange inside the buffer. This may leave the tail of the buffer unannotated.
+
+            return _End; // Using unaligned end will snap it to previous ASan granularity boundary.
+        }


+#ifdef __SANITIZE_ADDRESS__
+#include <cassert>
+#include <cstdio>


+        NO_SANITIZE_ADDRESS unsigned char* shadow_addr_of(const void* const addr) {
+            return reinterpret_cast<unsigned char*>(
+                (reinterpret_cast<uintptr_t>(addr) >> 3) + __asan_shadow_memory_dynamic_address);
+        }
+
+        NO_SANITIZE_ADDRESS unsigned char shadow_byte_of(const void* addr) {
+            return *shadow_addr_of(addr);
+        }


+        void print_shadow_bytes(const void* addr, size_t num_bytes, const void* error_addr = nullptr,
+            unsigned char expected_shadow_byte = 0xff /*unused shadow byte*/) {
+            constexpr size_t shadow_bytes_per_line  = 16;
+            constexpr uintptr_t bytes_per_line_mask = (shadow_bytes_per_line * shadow_granularity) - 1;
+


+    // Try push/pop at various sizes to cover resize code path (gh-6276)
+    for (size_t i = 1; i < Size; ++i) {
+        vector<T, Alloc> v(Size, T());
+        v.push_back(T());
+        assert(verify_vector(v));
+    }
+
+    for (size_t i = 1; i < Size; ++i) {
+        vector<T, Alloc> v(Size, T());
+        v.pop_back();
+        assert(verify_vector(v));
+    }
+}


        assert(verify_string(copy_assigned_sso_to_large));

-        str copy_assigned_large_to_large(get_large_input<CharType>());
+        str copy_assigned_large_to_large(get_large_input<CharType>()); // creating allocator 28 with arena 8


        assert(verify_string(move_assigned_sso_to_large));

-        str move_assigned_large_to_large(get_large_input<CharType>());
+        str move_assigned_large_to_large(get_large_input<CharType>()); // creating allocator 42 with arena 12


Modify vector and string annotations to extend 'last valid' point to …

f3cb662

…'end' point in situations where the end has been extended to the end of the allocation block.

Copilot AI review requested due to automatic review settings May 13, 2026 23:35

amyw-msft requested a review from a team as a code owner May 13, 2026 23:35

github-project-automation Bot added this to STL Code Reviews May 13, 2026

github-project-automation Bot moved this to Initial Review in STL Code Reviews May 13, 2026

Copilot started reviewing on behalf of amyw-msft May 13, 2026 23:36 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread stl/inc/xstring Outdated

amyw-msft added 2 commits May 14, 2026 09:38

Properly mirror xstring/vector changes.

17f94e2

Fix formatting

4b35e62

Copilot AI review requested due to automatic review settings May 14, 2026 17:13

Copilot AI reviewed May 14, 2026

View reviewed changes

Copilot started reviewing on behalf of amyw-msft May 14, 2026 17:23 View session

davidmrdavid reviewed May 14, 2026

View reviewed changes

Comment thread tests/std/tests/GH_006276_annotation_poison_cleanup/test.cpp Outdated

StephanTLavavej added bug Something isn't working ASan Address Sanitizer labels May 14, 2026

StephanTLavavej moved this from Initial Review to Work In Progress in STL Code Reviews May 14, 2026

Fix shadowing

7b5cd63

zacklj89 reviewed May 18, 2026

View reviewed changes

amyw-msft added 4 commits June 3, 2026 15:48

Remove gh6276 specific test since it's handled by other tests now.

c395821

Remove extra spaces, reorder headers, etc.

db3ced9

Apply clang-format

4663fc2

Copilot AI review requested due to automatic review settings June 4, 2026 20:43

Copilot started reviewing on behalf of amyw-msft June 4, 2026 20:43 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Conversation

amyw-msft commented May 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

davidmrdavid left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

StephanTLavavej commented May 14, 2026

Uh oh!

zacklj89 left a comment

Choose a reason for hiding this comment

Uh oh!

zacklj89 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

zacklj89 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

zacklj89 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

zacklj89 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

amyw-msft Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

amyw-msft Jun 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants