Octopii - Turn any Rust struct into a replicated, fault tolerant cluster

142 points Ok_Marionberry8922 59 comments

I’ve been working on Octopii for around a year now, a "batteries-included" library that aims to make building distributed systems in Rust as easy as writing a standard struct.

Usually, if you want to build a distributed Key Value store or a game server, you have to wire up a consensus engine (like Raft), build a networking layer, handle disk persistence, and pray you didn't introduce a race condition that only shows up in production.

Octopii acts like a "Distributed Systems Kernel." It handles the physics of the cluster (storage, networking, leader election) so you can focus entirely on your application logic.

You define a struct (your state) and implement a single trait. Octopii replicates that struct across multiple servers and keeps them consistent, even if nodes crash or hard drives fail.

// 1. Define your state
struct Counter { count: u64 }

// 2. Define your logic
impl StateMachineTrait for Counter {
    fn apply(&self, command: &[u8]) -> Result<Bytes, String> {
        // This runs deterministically on the Leader
        self.count += 1; 
        Ok(Bytes::from(self.count.to_string()))
    }
    // Octopii handles the disk persistence, replication, and networking automatically.
}

It’s effectively the infrastructure behind something like Cloudflare Durable Objects, but packaged as a crate you can run on your own hardware.

Under the Hood

I tried to take the "hard mode" route to ensure this is actually production ready, not just a toy, for that I implemented a Deterministic simulation testing:

  • The "Matrix" Simulation: Inspired by FoundationDB and Tigerbeetle, the test suite runs inside a deterministic simulator (virtual time, virtual network, virtual disk). I can simulate power failures mid-write ("torn writes") or network partitions to prove the database doesn't lose data.
  • Hardware-Aware Storage: includes walrus,a custom append only storage. It detects Linux to use io_uring for batching
  • The "Shipping Lane": It uses QUIC (via quinn) to multiplex connections. Bulk data transfer (like snapshots) happens on a separate stream from consensus heartbeats, so sending a large file never crashes the cluster.

Repository: https://github.com/octopii-rs/octopii

I’d love for you to try breaking it (or reading the simulation code) and let me know what you think :)