Rust vs Go Issue
I had an idea the other day, Go's threads are relatively cheap to spawn since they're not real kernel threads. So I made a multithreaded FS traverser in Go, and it worked well. It seemed to outperform du by 3x.
Then I thought, why not write it in Rust. So I did and I'm using use std::sync::mpsc::channel as the channels and regular thread::spawn for the threads. Although it's not quite the same, I thought it'd be cool to see the difference. However for larger directories, I get the following error in the Rust implementation:
Too many open files (os error 24)
But that doesn't happen in my Go implementation, on the exact same directory. So I'm guessing Rust's fs::read_dir keeps the file descriptors open.
Is there a way to fix this? or is the way I'm doing it just not possible?
Go impl: https://github.com/Cybergenik/godex
Rust impl: https://github.com/Cybergenik/rudex