How to use async Rust for non-IO tasks?

I'm working on a project that requires lots of small simulations to share a big shared resource: a GPU. Each similation repeatedly has a small job it wants to run, but it's more efficient if these jobs are batched before being sent off. Each simulation also has some CPU runtime cost to prepare the next job, so these should be run on a small threadpool. The GPU processing is a lot like IO, the CPU just has to sit around and wait for the result.

For some sense of scale, there are about 8 CPU cores running a total of 4000 jobs, and the GPU thread waits until there are 1024 jobs ready to go. I've got this system to work by manually writing out state machines and a bunch of crossbeam channel communication, but the code is a bit of a mess to say the least.

This seems exactly what async was designed for: multiplexing lots of small tasks onto a couple of threads in order to hide "IO" latency, while keeping the code almost as readable as blocking code.

I'm not sure how to actually go about implementing this: Most resources I find are about starting to use eg. tokio, its build-in threadpool and io worker threads, its existing futures etc. I'm not sure this is what I actually need, since I'm not doing any real IO. I should at least write my own Future, probably Waker, but do I need to write a thread pool that runs the tasks from scratch as well?

In short, I'm looking for some resources or a starting point on how to set up a more custom async system.

Edit:

It turns out everything was a lot simpler than I expected. tokio has tokio::sync::oneshot::channel, which easily allows you to communicate between a simple blocking thread that handles the GPU execution, and the async world. This means it's not necessary to write a custom runtime, or even a Waker or Future at all. The futures crate has something similar if you want to avoid using tokio.

This small demo seems to work great: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=23d94a7a29da67ed1bcb192e5a5830a7. I use the runtime builder so I can opt-out of the default IO and time theads tokio spawns.

This still requires a dedicated thread to block on the GPU and job collection, but that's fine since in practice you want only one or a few threads communicating with the GPU anyway.

Thanks everyone for the help!