[Rust] array::transform::tests::test_struct failed

[asfimport]

Test array::transform::tests::test_struct  in arrow/src/array/transform/mod.rs failed when swap the first two elements:

change from

// code placeholder
let strings: ArrayRef = Arc::new(StringArray::from(vec![
    Some("joe"),
    None,

to

// code placeholder
let strings: ArrayRef = Arc::new(StringArray::from(vec![
    None,
    Some("joe"),

The failure was first found when I report #27069

 

 

Reporter: Qingyou Meng / @mqy
Assignee: Jorge Leitão / @jorgecarleitao

Related issues:

• [Rust]: Comparison of list arrays with differing offsets fails (relates to)
• [Rust] pitfall in xxx_equal functions (is related to)

PRs and other links:

• GitHub Pull Request #9193
• GitHub Pull Request #9211

_{Note: This issue was originally created as ARROW-11239. Please see the migration documentation for further details.}

[asfimport]

Qingyou Meng / @mqy:
@jorgecarleitao

Test passed if construct the expect directly (not from slice()).

look like the problem is caused by ArrayRef::slice(), which contains cloned buffers and null bitmap.

1. Add new array_data_slice_problem to verify failure case 1:

    #[test]
    fn array_bug_1() {
        let strings: ArrayRef = Arc::new(StringArray::from(vec![
            None,
            Some("b"),
        ]));
        let strings_slice = strings.slice(1, 1);
        let strings2: ArrayRef = Arc::new(StringArray::from(vec![
            Some("b"),
        ]));
        assert_eq!(&strings2, &strings_slice, "\n##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
    }

Failed with error:

thread 'array::transform::tests::array_bug_1' panicked at 'assertion failed: `(left == right)`
  left: `StringArray
[
  "b",
]`,
 right: `StringArray
[
  "b",
]`: 
##1: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 0, buffers: [Buffer { data: Bytes { ptr: 0x7fd413704600, len: 8, data: [0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fd413704700, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fd413704680, len: 1, data: [1] }, offset: 0 } }) }
##2: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 1, buffers: [Buffer { data: Bytes { ptr: 0x7fd413704300, len: 12, data: [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fd413704400, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fd413704380, len: 1, data: [2] }, offset: 0 } }) }', arrow/src/array/transform/mod.rs:823:9

 

2. Adjust array_data_slice_problem, add element Some("a"):

 

    #[test]
    fn array_bug_2() {
        let strings: ArrayRef = Arc::new(StringArray::from(vec![
            Some("a"),
            None,
            Some("b"),
        ]));
        let strings_slice = strings.slice(2, 1);
        let strings2: ArrayRef = Arc::new(StringArray::from(vec![
            Some("b"),
        ]));
        assert_eq!(&strings2, &strings_slice, "\n##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
    }


Panic on call to `strings.slice(2, 1)`:

thread 'array::transform::tests::array_bug_2' panicked at 'range end index 2 out of range for slice of length 0', arrow/src/array/equal/utils.rs:81:49

 

[asfimport]

Jorge Leitão / @jorgecarleitao:
great finding. cc [~nevi_me]

[asfimport]

Andrew Lamb / @alamb:
Thanks @mqy – this is a really nice one find and well written ticket

[asfimport]

Jorge Leitão / @jorgecarleitao:
Issue resolved by pull request 9211
#9211