Rust program is slower than equivalent Zig program

178 points SaltyMaybe7887 117 comments

I’m trying out Rust for the first time and I want to port something I wrote in Zig. The program I’m writing counts the occurences of a string in a very large file after a newline. This is the program in Zig:

const std = @import("std");

pub fn main() ! void {
    const cwd = std.fs.cwd();
    const file = try cwd.openFile("/lib/apk/db/installed", .{});
    const key = "C:Q";

    var count: u16 = 0;

    var file_buf: [4 * 4096]u8 = undefined;
    var offset: u64 = 0;

    while (true) {
        const bytes_read = try file.preadAll(&file_buf, offset);

        const str = file_buf[0..bytes_read];

        if (str.len < key.len)
            break;

        if (std.mem.eql(u8, str[0..key.len], key))
            count +|= 1;

        var start: usize = 0;
        while (std.mem.indexOfScalarPos(u8, str, start, '
')) |_idx| {
            const idx = _idx + 1;
            if (str.len < idx + key.len)
                break;
            if (std.mem.eql(u8, str[idx..][0..key.len], key))
                count +|= 1;
            start = idx;
        }

        if (bytes_read != file_buf.len)
            break;

        offset += bytes_read - key.len + 1;
    }
}

This is the equivalent I came up with in Rust:

use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};

fn main() -> io::Result<()> {
    const key: [u8; 3] = *b"C:Q";

    let mut file = File::open("/lib/apk/db/installed")?;
    let mut buffer: [u8; 4 * 4096] = [0; 4 * 4096];
    let mut count: u16 = 0;

    loop {
        let bytes_read = file.read(&mut buffer)?;

        for i in 0..bytes_read - key.len() {
            if buffer[i] == b'
' && buffer[i + 1..i + 1 + key.len()] == key {
                count += 1;
            }
        }

        if bytes_read != buffer.len() {
            break;
        }

        _ = file.seek(SeekFrom::Current(-(key.len() as i64) + 1));
    }

    _ = count;

    Ok(())
}

I compiled the Rust program with rustc -C opt-level=3 rust-version.rs.
I compiled the Zig program with zig build-exe -OReleaseSafe zig-version.zig.

However, I benchmarked with hyperfine ./rust-version ./zig-version and I found the Zig version to be ~1.3–1.4 times faster. Is there a way I can speed up my Rust version?

The file can be downloaded here.

Update: Thanks to u/burntsushi, I was able to get the Rust version to be a lot faster than the Zig version. Here is the updated code for anyone who’s interested (it uses the memchr crate):

use std::os::unix::fs::FileExt;

fn main() -> std::io::Result<()> {
    const key: [u8; 3] = *b"C:Q";

    let file = std::fs::File::open("/lib/apk/db/installed")?;

    let mut buffer: [u8; 4 * 4096] = [0; 4 * 4096];
    let mut count: u16 = 0;
    let mut offset: u64 = 0;

    loop {
        let bytes_read = file.read_at(&mut buffer, offset)?;

        let finder = memchr::memmem::Finder::new("
C:Q");
        count += finder.find_iter(&buffer).count() as u16;

        if bytes_read != buffer.len() {
            break;
        }

        offset += (bytes_read - key.len() + 1) as u64;
    }

    _ = count;

    Ok(())
}

Benchmark:

Benchmark 1: ./main
  Time (mean ± σ):       5.4 ms ±   0.9 ms    [User: 4.3 ms, System: 1.0 ms]
  Range (min … max):     4.7 ms …  13.4 ms    213 runs
 
Benchmark 2: ./rust-version
  Time (mean ± σ):       2.4 ms ±   0.8 ms    [User: 1.2 ms, System: 1.4 ms]
  Range (min … max):     1.3 ms …  12.7 ms    995 runs
 
Summary
  ./rust-version ran
    2.21 ± 0.78 times faster than ./main