[Swift] Swift Arrow IPC stream implementation is not in sync with other languages

[abandy]

Describe the bug, including details regarding any error messages, version, and platform.

@ianmcook found a discrepancy between the Swift Arrow Flight behavior and python. Swift is expecting the footer to contain the schema in the footer and this is not consistent with other languages. This needs to be updated for Swift.

Component(s)

Swift

[ianmcook]

To clarify: the discrepancy is in the Arrow IPC stream format, not in Arrow Flight.

[abandy]

Hmm, there are a couple of tests in ArrowTests/IPCTests.swift that pull in files generated by a go script (swift/data-generator/swift-datagen/main.go). The way I understood the Arrow IPC format is that it is the same as the File format except for the file marker (that is the way the Swift implementation is coded). Is this not true?

[ianmcook]

There's a description of the differences here:
https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format
https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format

The IPC stream format lacks the magic numbers and the footer that the IPC file format has.

There's a file containing a valid Arrow IPC stream here if you want to use it for testing: https://github.com/apache/arrow-experiments/tree/main/data/rand-many-types

(This is confusing because you can write an IPC stream to a file, which is what I did there. But it's in the stream format, not the file format.)

[abandy]

Aha, thank you! I will update accordingly!