TMF: compress your meshes by 86-89%, blazingly fast!
TMF: compress your meshes by 86-89%, blazingly fast! TMF is a file compression format written fully in Rust that achieves both high compression ratios, and insane decode speeds. It also aims to provide a user-friendly, explicit and well-documented API.
What did this update(0.2) bring:
- File size cut by half!
- Single-threaded read speed increased \~3x times!
- Support for decoding a mesh on multiple threads at once, which in some test cases cut decode time by further 2.8 times!
- Support for mesh tangent data.
- Support for custom mesh data! Add named mesh data, which is too, automatically compressed! Currently supports: IndexArrays, Intiges, Floats, and RGBA colors.
- Both the TMF reader and writer have been completely overhauled, and extensively tested by fuzzing, so reader should never crash!
How good is tmf at compressing?
Here is some comparisons with different model formats, comparing tmf to a wide range of other formats: Model used for comparisons has 15.7 k triangles, and it is available as sunsan.obj in testing folder of the repo
| Format | Size |
|---|---|
| .obj | 1.3 MB |
| zip(deflate) compressed .obj | 367.7 kB |
| .fbx | 651.0 kB |
| zip(deflate) compressed .fbx | 600.6 kB |
| .gltf | 476.5 kB |
| zip(deflate) compressed .gltf | 302.1 kB |
| .glb | 356.6 kB |
| zip(deflate) compressed .glb | 267.5 kB |
| .tmf | 308.3 kB |
| .tmf with pre-encode optimizations enabled | 165.5 kB |
| .tmf with pre-encode optimizations and hand-picked quality settings | 147.3 kB |
| zip(deflate) compressed .tmf | 307.9 kB |
| zip(deflate) compressed .tmf, with pre-encode optimizations | 162.8 kB |
| zip(deflate) compressed .tmf, with pre-encode optimizations, hand-picked quality settings | 145.0 kB |
| draco on max compression settings | \~22 kB |
As you can see, a .tmf mesh is smaller than almost all the other formats, except for Google Draco. Draco is still about 5x better at compression, but tmf is now far closer(previously it was \~10x worse than Draco). The situation might seem clear-cut, but there is one more metric to consider: decode speed.
How fast is tmf?
TMF is crazy-fast. Decoding a mesh with 15.7K triangles takes just 675.85 µs(on a single thread), that is 0.675 ms! At its size of 165.5 kB, we could estimate decode speed to be roughly 244 MB/s, which means that for HDDs and many cheaper SSDs, you could decode data faster than you could read it! What is even better, it is so fast, that on most systems it would take considerably more time to read an uncompressed 3D model, than it would take to read and decode a.tmf file! Comparing to Draco again, decoding this same exact mesh took Draco around 7-10 ms, which means that tmf is on average about 10 to 14 times faster than Draco! If we were to take the 10 times figure, that would mean that while Draco compresses meshes 5x better, it also decodes them 10x slower than tmf. Of course, we should take this with a pinch of salt. I have little experience using Draco, so it is entirely possible I just used it improperly. It is still, however, very rewarding to see my project come even close to something made by a team of people at Google.
Thanking my contributors
This was also the very first of my Rust projects that had someone else contribute. So I would like to thank people who have contributed:
https://github.com/nicopap \- various small, but significant QOL improvements to the API and suggestions for features
https://github.com/CptPotato \- for work on better encoding for normals (not merged yet, so something to look forward to in 0.3!)
Links:
https://crates.io/crates/tmf
https://github.com/FractalFir/tmf
Any questions?
If you have any questions, feel free to ask. I will try to answer all of them.