Optimizing a One Billion Row Challenge in Rust with Polars

146 points matt78whoop

I saw this Blog Post on a Billion Row challenge for Java so naturally I tried implementing a solution in Rust using mainly polars.
Code/Gist here

Running the code on my laptop, which is equipped with an i7-1185G7 @ 3.00GHz and 32GB of RAM, I achieved a runtime of approximately 4.5 seconds within a Dev Container.

Any suggestions for further optimizing the solution?

Edit:
I missed the requirements that is must be implemented using only the Standard Library and in Alphabetical order, here is a table of both implementations!

Implementation Time Code/Gist Link
Rust + Polars 6.5s https://gist.github.com/Butch78/702944427d78da6727a277e1f54d65c8
Rust STD Libray 18.5s https://gist.github.com/Butch78/893561f77de456a096ed6d1e672c4bed