Hot take: PhantomData should not be necessary

I can’t help but feel like PhantomData, in its most typical usage (i.e. just to get the compiler to stop complaining about its absence for a given type parameter) just should not be necessary at all. I can get it for other usages, for example when you want to bind a lifetime parameter and a type parameter (i.e. a phantom &’a mut T) without actually taking and storing a reference, as this is actually providing useful information and functionality both to the developer and compiler. But adding phantom data just so a type can use a T without having one stored feels like an implementation detail that should’ve never leaked out of the compiler to the developer. Please do correct me if I’m wrong, as I’m hardly a Rust expert, but what would the potential dangerous implications be of the compiler inferring an unused T being a PhantomData implicitly? What’s stopping us from forgoing this constraint?