Rewrote our message routing in rust and holy shit

We had this python service handling message routing between our microservices, it worked fine at low volume but started choking around 5k messages per second. Cpu usage was insane and we were seeing latency spikes.

Decided to rewrite the hot path in rust as an experiment. It took about a week to get it working with the same logic. Deployment day came and we werre watching metrics.

Python version: 5k msg/s, 80% cpu, 200ms p99 latency

Rust version: 45k msg/s, 12% cpu, 8ms p99 latency

I know rust has a reputation for being hard but for this kind of systems programming it turmed out to be absolutely worth it. The borrow checker is annoying at first because it won't let you compile until you handle memory properly, but once it compiles you know it won't have data races or memory leaks.

Our stack is pretty standard now. Postgres for persistence, redis for caching, nats handling the message broker since its very lightweight and still handles the output, and of course this rust service doing all the routing logic. Whole thing is like 2000 lines and handles our entire production load on a single small instance.