Octopii - Turn any Rust struct into a replicated, fault tolerant cluster
I’ve been working on Octopii for around a year now, a "batteries-included" library that aims to make building distributed systems in Rust as easy as writing a standard struct.
Usually, if you want to build a distributed Key Value store or a game server, you have to wire up a consensus engine (like Raft), build a networking layer, handle disk persistence, and pray you didn't introduce a race condition that only shows up in production.
Octopii acts like a "Distributed Systems Kernel." It handles the physics of the cluster (storage, networking, leader election) so you can focus entirely on your application logic.
You define a struct (your state) and implement a single trait. Octopii replicates that struct across multiple servers and keeps them consistent, even if nodes crash or hard drives fail.
// 1. Define your state
struct Counter { count: u64 }
// 2. Define your logic
impl StateMachineTrait for Counter {
fn apply(&self, command: &[u8]) -> Result<Bytes, String> {
// This runs deterministically on the Leader
self.count += 1;
Ok(Bytes::from(self.count.to_string()))
}
// Octopii handles the disk persistence, replication, and networking automatically.
}
It’s effectively the infrastructure behind something like Cloudflare Durable Objects, but packaged as a crate you can run on your own hardware.
Under the Hood
I tried to take the "hard mode" route to ensure this is actually production ready, not just a toy, for that I implemented a Deterministic simulation testing:
- The "Matrix" Simulation: Inspired by FoundationDB and Tigerbeetle, the test suite runs inside a deterministic simulator (virtual time, virtual network, virtual disk). I can simulate power failures mid-write ("torn writes") or network partitions to prove the database doesn't lose data.
- Hardware-Aware Storage: includes walrus,a custom append only storage. It detects Linux to use
io_uringfor batching - The "Shipping Lane": It uses QUIC (via
quinn) to multiplex connections. Bulk data transfer (like snapshots) happens on a separate stream from consensus heartbeats, so sending a large file never crashes the cluster.
Repository: https://github.com/octopii-rs/octopii
I’d love for you to try breaking it (or reading the simulation code) and let me know what you think :)