ocrs - A new open source OCR engine, written in Rust

I have released an early preview of ocrs, a new open source OCR engine that is "end-to-end Rust" (for inference at least, model training uses PyTorch). The goal is to make an easy to use, portable and embeddable OCR engine, trained on openly licensed datasets.

I previously worked on tesseract-wasm, a WebAssembly build of the popular Tesseract library (written in C++, maintained at one time by Google). Tesseract works quite well on clean, straight document images with simple layouts, but often fails to detect text in more varied images (think photos, artwork, screenshots with text overlaid, complex layouts etc). This is due to having parts of the OCR pipeline using hand-coded heuristics which tend to be brittle. It also represents coordinates as axis-aligned bounding boxes and thus does not supported rotated text well.

OCR is a well studied problem and there are many commercial services and open source projects (eg. EasyOCR) that have improved upon this by going in a more Software 2.0 direction. Nevertheless Tesseract is still the de-facto open source library because it is portable, embeddable and usable from many languages. I think there is an opportunity to create something better with Rust (for inference) + PyTorch (for training) + modern datasets.

ocrs is initially available as a Rust library and CLI tool. Example CLI usage:

cargo install ocrs-cli

# Extract text, print to stdout
ocrs image.png

# Extract text, output text + layout info as JSON
ocrs image.png --json -o output.json

# Annotate image, showing location of detected text
ocrs image.png --png -o annotated.png

Recognition quality is very much "alpha" and there is a lot of iteration to be done on the models before it can be a general replacement for Tesseract or other OCR engines. That is going to keep me busy for the next few months. Nevertheless, it already works better for some kinds of inputs.