Benchmarked std::simd vs hand-written NEON on an M4. My first numbers were wrong.

A while back I benchmarked Rust’s std::simd against hand-written NEON intrinsics on an M4. 9 scenarios, each written three ways: scalar, std::simd, NEON.

I published it and said std::simd was up to 7.7x slower than scalar in some cases, because the portable API can’t express things like vld3q_u8 or vhaddq_s16.

That was wrong.

Some of it was just bugs. My RGB to grayscale NEON number was 4x, someone ran cargo bench and got about 1x. Most of the “8x slower” std::simd results came from scalar loops (RGB deinterleave, i16 to i32) sitting inside the SIMD path. People sent fixes with simd_swizzle! and .cast() and those loops went away. There was also a windows(9) in the sorted check doing 8x more comparisons than needed.

After fixing that there was still a gap, and the reason was that the two versions weren’t actually the same code. std::simd used f32x8 for the dot product, NEON used one f32x4 accumulator. Byte counting was u8x32 vs u8x16. I was comparing vector width and accumulator count, not the API.

A few things I found while lining them up:

f32x8 on a 128-bit ISA is two f32x4 registers, so you get two FMA chains for free. One NEON accumulator is latency-bound. Add a second one and it matches.

Mask::<u8, 32>::to_bitmask() has no single NEON instruction. No movemask on ARM, so 16 lanes ended up faster than 32.

vextq_s32(prev, cur, 3) is exactly what simd_swizzle! does for the shifted compare in the sorted check.

In the audio scenarios, the scalar baseline was already auto-vectorized to shadd. All three versions were the same instruction. That benchmark measured nothing.

And one .enumerate() on a loop running near one iteration per cycle cost 13% by itself.

Once vector width, accumulator count and reduction frequency were the same, std::simd and NEON were within a few percent in every scenario.

Writeup: https://github.com/Erio-Harrison/simd_benchmark/blob/master/BLOG.md
Repo: https://github.com/Erio-Harrison/simd_benchmark

Thanks to everyone who read the code and told me what was wrong. Credits are in the post.