Introducing: Depends
TLDR: new dependency graph/incremental computation library for arbitrary rust structs
One of the most elegant solutions to a technical problem I've encountered was during a previous stint in HFT, where many of the trading teams would build strategies incrementally by composing small stateful units of logic in a dependency graph structure.
The main benefits of this approach are:
- Isolation of state - data is owned by a node which can mutate it internally but only pass shared references to dependents.
- Isolation of logic - users are encouraged to break down complex apps in to small, composable, testable transformations.
- Performance/caching - only nodes which have children which have changed values need to be recomputed, resulting in free incremental computation.
- Flexibility/maintainability - core application logic is declarative, as all dependencies (nodes + edges) are clearly defined. This generally reduces the number of 'surprises' when performing invasive surgery.
So, for areas where multiple external data sources can change, and you want to reliably and deterministically respond to these events, dependency graph structures are a great tool.
There are some excellent crates already in the Rust ecosystem (notably Salsa) for performing incremental computation, although they generally have different goals to Depends.
Depends is a library which is tailored around cases where:
- You construct graphs relatively infrequently, compared to updating/resolving them.
- Your problem can be a structured as a DAG (what are all the actions/results I would like given the latest data).
- You have multiple 'inputs' (or leaf nodes) which may change by external processes at any point in time.
- Performance is a requirement (using shared references over cloning and avoiding dynamic dispatch).
You can think about this more generally as a library which ‘glues’ together arbitrary Rust structs and transformations, and recalculates those struct’s state on demand, when necessary.
Many applications can benefit from an incremental computation approach. Check out the getting started guide to see if it fits yours.
Contributions/feedback welcome!