How to optimize really large hashmaps?

Recently I tuned two programs: a Rust program that uses std::collections::HashMap / hashbrown, and a C++ program that uses absl::flat_hash_map. Both of them operate hashmaps in single threads and the hashmap sizes are larger than 10 MiB. According to the profile result, hashmap operations consumed the majority of the time. So I'm wondering if there is a way to optimize them.

High hash function quality implies perfect random distribution. Presumably, on large hashmaps, perfect random distribution implies an extremely high cache miss rate. To verify this assumption, I conducted a benchmark, showing the huge impact of hashmap sizes on the performance.

I made some attempts, trying to use instruction-level parallelism to hide the latency of memory accesses, but all failed.