1BRC in safe Rust

63 points Ka1kin

My latest toy project is a safe Rust implementation of the One Billion Row Challenge.

Most others depend on a judicious bit of unsafe here or there (memory mapped files and the like), and have quite an array of external dependencies. I'm not actually against that, but I wanted to see where I could get without it. I did pull in rustc-hash because the small performance boost was worthwhile (about 10% of the CPU time appears to be spent hashing city names), but otherwise, it's all std.

I was delighted to discover Read::take just when I was getting ready to write my own.

Most of the implementation time and complexity is around handling inconvenient page boundaries while avoiding unnecessary copies and allocations.

Performance-wise, it competes well with other implementations that I've seen, running in under 3.2s on my M2 Max MBP.