My first Rust project: an offline manga translator with candle ML inference

Hi folks,

Although it's still in active development, I've got good results to share!

It's an offline manga translator that utilizes several computer vision models and LLMs. I learned Rust from scratch this year, and this is my first project using pure Rust. I spent a lot of time tweaking the performance based on CUDA and Metal (macOS M1, M2, etc.).

This project was initially used ONNX for inference, but later re-implemented all models in candle to achieve better performance and control over the model implementation. You may not care, but during development, I even contributed to the upstream libraries to make them faster.

Currently, this project supports vntl-llama3-8b-v2, lfm2-350m-enjp-mt LLM for translating to English, and a multilingual translation model has been added recently. I would be happy if you folks could try it out and give some feedback!

It's called Koharu, the name comes from my favorite character in a game; you can find it here: https://github.com/mayocream/koharu

I know there already are some open-source projects using LLM to translate manga, but from my POV, this project uses zero Python stuff; it's another try to provide a better translation experience.