cargo-mutants 0.2: a new mutation testing tool
tl;dr: cargo install cargo-mutants; cd ~/src/myproj; cargo mutants
cargo-mutants is a new take on mutation testing in Rust. Mutation testing is a technique for finding gaps in your program's test suite, by mechanically generating buggy versions of the program, and then seeing if the tests catch the bugs.
In cargo-mutants the mutations are generated by using syn to parse the Rust source and understand the return types of all functions, and then replacing function bodies with a return of a value that will (hopefully) pass the typecheck and compile, but be wrong. For example, any function that returns a bool could be mutated to do nothing but return true. Good tests ought to notice this, unless you really do have a function that intentionally only ever returns true.
cargo-mutants requires no changes to the source tree: just install and run it on any(\*) Rust source tree that builds with Cargo and has hermetic tests. All the mutations are applied in a copy of the tree. (Actually there are a few cases that will trip it up at the moment, including source that's not in ./src/**/*.rs, and relative-path dependency overrides. But most trees should work.)
Mutation testing can complement looking at test coverage: coverage tells you which lines or blocks were reached, but mutation coverage tells you something about whether the test results actually depend on the code.
There's still more that can be done here but I'd love to hear any feedback, whether that is interesting bugs or coverage gaps that it caught, or trees where it doesn't work well.
