What is the point of using async?

70 points Bulb211

I know people say it is more efficient, but I honestly have problem seeing any reason why, in a well optimized operating system, it should. So let's focus on Linux.

Comparing async code and corresponding sync code:

  • Every switch between tasks is a system call, so it takes a kernel entry and exit in either case. And while the sync (more threads) case involves returning to a different thread, it is still the same address space, so no expensive TLB flush needed in either case.
    • The Linux scheduler has been O(1) for ages, so it does not matter whether the schedule() call that runs during the kernel exit is dealing with 10 or 10 000 threads.
  • In fact the async code involves additional kernel entries and exits to the epoll or similar function, so there should be more overhead there, not less.
  • If we are after performance, the tasks run in multi-threaded runtime and so any tasks that modify shared data still have to be synchronized.
    • This is of course the reason why JavaScript runtimes use async, but that is because it can's either ensure the tasks are independent nor make the low-level operations atomic cheaply, which is why it runs single-threaded. But Rust can track dependencies and the runtimes are generally multi-threaded, so this reason does not apply.
  • While each thread needs virtual address range allocated for its stack, the actual memory is provided on demand. So while it does use a bit more memory, it shouldn't be too bad.
    • Let's use a thread pool that reuses the threads to avoid allocating and freeing the kernel task structures (that weigh 8 KiB) and stack memory ranges in a short loop; that would indeed be an extra cost.

Additionally,

  • Not all system calls have async variants. For example reading files is synchronous, and on a server it even often goes over the same network as the whatever requests we are handling.
  • And due to the way virtual memory works any instruction can (occasionally) need to wait for disk I/O, so all code may (rarely, but still) block.

So when and why is an async runtime actually more efficient? In Linux. Or, another system if it has more efficient threads than Linux.

Also I can understand how async simplifies event-loop-based code in things like minimal embedded runtimes where you don't have and don't want to implement full preemptive threads. But again, that does not really affect Linux, which has perfectly serviceable threads.