Why ELF is bigger than EXE?

I don't know if this is a Rust-specific thing, but I noticed all releases of b3sum have binary sizes where GNU-Linux executable is many ORDERS OF MAGNITUDE bigger than the Windows exec, while macOS is "sitting in the middle". Even my xorsum program had this difference after configuring the release profile to do LTO, strip symbols, and codegen 1.

Something interesting happens depending on the linker. xorsum.exe compiled with MSVC (no cross-compiling) is slightly smaller than MinGW (when cross-compiling from Linux), why does this happen?