ripwc: a much faster Rust rewrite of wc – Up to ~49x Faster than GNU wc

https://github.com/LuminousToaster/ripwc/

Hello, ripwc is a high-performance rewrite of the GNU wc (word count) inspired by ripgrep. Designed for speed and very low memory usage, ripwc counts lines, words, bytes, characters, and max line lengths, just like wc, while being much faster and has recursion unlike wc.

I have posted some benchmarks on the Github repo but here is a summary of them:

• 12GB (40 files, 300MB each): 5.576s (ripwc) vs. 272.761s (wc), ~49x speedup.
• 3GB (1000 files, 3MB each): 1.420s vs. 68.610s, ~48x speedup.
• 3GB (1 file, 3000MB): 4.278s vs. 68.001s, ~16x speedup.

How It Works:

• Processes files in parallel with rayon with up to X threads where X is the number of CPU cores.
• Uses 1MB heap buffers to minimize I/O syscalls.
• Batches small files (<512KB) to reduce thread overhead.
• Uses unsafe Rust for pointer arithmetic and loop unrolling

Please tell me what you think. I'm very interested to know other's own benchmarks or speedups that they get from this (or bug fixes).

Thank you.

Edit: to be clear, this was just something I wanted to try and was amazed by how much quicker it was when I did it myself. There's no expectation of this actually replacing wc or any other tools. I suppose I was just excited to show it to people.