Bizarre memory leak caused by tokio runtime
I have a code that drops and creates huge object in memory once in a while. And yesterday I've found out that it leaks memory. Today I've managed to create a simple program with the similar memory problems. So since it is not very big, I will post it here:
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Mutex;
pub struct Object {
pub a: Vec<u32>,
pub b: Vec<u32>,
pub c: u32,
pub d: u32,
pub e: u32,
pub f: Option<u128>
}
#[allow(unused)]
pub struct ObjectHolder {
objects: Vec<Arc<Object>>
}
impl ObjectHolder {
pub fn new() -> ObjectHolder {
let mut res = vec![];
for _ in 0..10_000_000 {
res.push(Arc::new(Object {
a: vec![12123, 123123, 12123, 12124, 1231512],
b: vec![12423, 234, 2342],
c: 423423,
d: 123123,
e: 12312,
f: Some(392932)
}))
}
return ObjectHolder {
objects: res
}
}
}
pub struct Main {
holder: Arc<Mutex<ObjectHolder>>
}
impl Main {
pub fn new() -> Arc<Main> {
return Arc::new(Main {
holder: Arc::new(Mutex::new(ObjectHolder::new()))
})
}
pub async fn build_objects(&self) {
let mut holder = self.holder.lock().await;
*holder = ObjectHolder::new();
println!("BUILT NEW OBJECTS");
}
}
#[tokio::main]
async fn main() {
let main = Main::new();
let cnt_iter = 100;
tokio::spawn(async move {
for _ in 0..cnt_iter {
println!("STARTIGN NEW CYCLE");
main.build_objects().await;
(0..100).for_each(|_| {
tokio::spawn(async move {
return Some(0)
});
});
println!("WAITING 10 SECONDS");
tokio::time::sleep(Duration::from_secs(10)).await;
}
}).await.unwrap();
}
Some of my comments and observations:
- This is definitely a memery leak. Because after 10-20 iterations the process is interrupted with SIGKILL.
- Removing outer
tokio::spawn(line 53) fixes memory leak. The memory usage is stable after first 2-3 iterations. - If I don't spawn 100 tasks inside each iteration(line 52-62), then there is no memory leak too.
- Storing
Objectinstead ofArc<Object>reduces the speed of memory loss, but the memory leak persists. - I've compiled this code with rust 1.62.0 and 1.64.0-nightly. In release and debug mods behavior is the same.
Upd: Updated Object . Looks like with this structure the memory ends faster for some reason(in previous version the Object contained a vector of size 30 filled with u32)