Drawbacks of Option<T>

In Rust, enums are the only way to express alternatives. Enums let you define tagged unions that are nominally distinct: two enums with the same set of alternatives are not interchangeable. I've also used Haskell in the past and while I was convinced types like Option/Maybe were the optimal solution for handling the idea of optionality, somehow I was always thinking that there was some sort of fallacy hidden in there somewhere.

A few months ago I came across the "Maybe Not" talk by Rich Hickey, the creator of Clojure. He talks about the difference between inbound and outbound optionality when it comes to versioning API changes. He boils down the compatibility of API changes to the idea of requirements and promises. When you make an argument optional, you're relaxing the requirements for calling the function, and this means that this should ideally be a non-breaking change. Similarly, when you realize that a function returning an optional value could now return a value in all cases, you're strengthening your promise by no longer returning something optional, and this should ideally also be a non-breaking change. Rich Hickey does a fantastic job explaining this https://youtu.be/YR5WdGrpoug?t=496

The problem is that in Rust, switching an argument or a return type between T and Option<T> is always breaking, even in cases where we're relaxing requirements or strengthening promises. This makes me wonder if the idea of using a sum type for expressing optionality could have been broken all along. In the talk, he says that the wording around these types is responsible for making you think that they're an appropriate abstraction, when in reality you're looking for "proper" unions (as opposed to tagged unions).

The thing is that languages that use "proper" unions don't seem to get it completely right either. In TypeScript or statically-typed Python, optional values are a union with None, and you lose some nuance since you can't make an optional None anymore.

I've come to the conclusion that reasoning with sets and proper unions of acceptable types is the right approach, but it's flawed when you represent optionality with an actual None value. Making a type optional means that it's the set of acceptable types itself that should become possibly empty instead of adding a sentinel value representing the optionality.

If we consider the actual sets of types, the "flawed" union with None results in a set of cardinality >= 2 {T, None} whereas the "proper" solution would result in a set of cardinality >= 0 {T?}. The concept of a None type would instead be some sort of "void" represented by the empty set {}, it wouldn't exist as an actual value.

I found a RFC about anonymous sum types https://github.com/rust-lang/rfcs/issues/294. The idea is that since Rust already has anonymous product types with tuples, anonymous sum types would implement this idea of "proper" unions. Maybe this would allow something like the "void" type I was just talking about by using an empty "sum tuple" (as opposed to the current "product tuple").

Another resource that made me think was this introduction to the Icon programming language https://www2.cs.arizona.edu/icon/intro.htm. It's not really focused on type systems but it deals with the idea of optionality in a similar way. I'm curious if there's anything else out there that would potentially comply with this idea of compatibility for relaxing requirements and strengthening promises, but without introducing any sentinel value.