Bizarre memory leak caused by tokio runtime

197 points ascending_fourth

I have a code that drops and creates huge object in memory once in a while. And yesterday I've found out that it leaks memory. Today I've managed to create a simple program with the similar memory problems. So since it is not very big, I will post it here:

use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Mutex;

pub struct Object {
    pub a: Vec<u32>,
    pub b: Vec<u32>,
    pub c: u32,
    pub d: u32,
    pub e: u32,
    pub f: Option<u128>
}

#[allow(unused)]
pub struct ObjectHolder {
    objects: Vec<Arc<Object>>
}

impl ObjectHolder {
    pub fn new() -> ObjectHolder {
        let mut res = vec![];
        for _ in 0..10_000_000 {
            res.push(Arc::new(Object {
                a: vec![12123, 123123, 12123, 12124, 1231512],
                b: vec![12423, 234, 2342],
                c: 423423,
                d: 123123,
                e: 12312,
                f: Some(392932)
            }))
        }
        return ObjectHolder {
            objects: res
        }
    }
}

pub struct Main {
    holder: Arc<Mutex<ObjectHolder>>
}

impl Main {
    pub fn new() -> Arc<Main> {
        return Arc::new(Main {
            holder: Arc::new(Mutex::new(ObjectHolder::new()))
        })
    }

    pub async fn build_objects(&self) {
        let mut holder = self.holder.lock().await;
        *holder = ObjectHolder::new();
        println!("BUILT NEW OBJECTS");
    }
}

#[tokio::main]
async fn main() {

    let main = Main::new();

    let cnt_iter = 100;

    tokio::spawn(async move {
        for _ in 0..cnt_iter {
            println!("STARTIGN NEW CYCLE");
            main.build_objects().await;

            (0..100).for_each(|_| {
                tokio::spawn(async move {
                    return Some(0)
                });
            });

            println!("WAITING 10 SECONDS");
            tokio::time::sleep(Duration::from_secs(10)).await;
        }
    }).await.unwrap();
}

Some of my comments and observations:

  1. This is definitely a memery leak. Because after 10-20 iterations the process is interrupted with SIGKILL.
  2. Removing outer tokio::spawn(line 53) fixes memory leak. The memory usage is stable after first 2-3 iterations.
  3. If I don't spawn 100 tasks inside each iteration(line 52-62), then there is no memory leak too.
  4. Storing Object instead of Arc<Object> reduces the speed of memory loss, but the memory leak persists.
  5. I've compiled this code with rust 1.62.0 and 1.64.0-nightly. In release and debug mods behavior is the same.

Upd: Updated Object . Looks like with this structure the memory ends faster for some reason(in previous version the Object contained a vector of size 30 filled with u32)