fastbloom: The fastest Bloom filter in Rust

123 points tomtomwombat

I spent the last few weeks working on my first Rust project: fastbloom.

fastbloom is a Bloom filter 50-1000% faster than existing Bloom filters implemented in Rust. This is because it uses L1 cache friendly blocks, efficiently derives many index bits from only one real hash per item, and leverages other research findings on Bloom filters.

fastbloom is implemented as a blocked Bloom filter. Blocked Bloom filters partition their underlying bit array into sub-array “blocks”. Bits set and checked from the item’s hash are constrained to a single block instead of the entire bit array. This allows for better cache-efficiency and the opportunity to leverage SIMD and SWAR operations when generating bits from an item’s hash.

Any feedback welcome.