PSA: Enum optimization for Vec/String can use the capacity's niche (beyond isize::MAX) rather than the null pointer niche.

176 points kmdreko 37 comments

Just came across this property that was new to me and seems seldom known so I figured I'd highlight it.

I had always "known" that the null pointer niche optimization existed for Vec/String such that Option<Vec<_>> would be the same size as Vec<_> because the None variant could fit the null pointer niche not used by the Vec. And I figured that was it, any more elaborate enums like this

enum MyEnum {
    A(String),
    B,
    C,
    D,
    E,
}

would be forced into using a separate discriminant field since there were more than one other variant to distinguish from. And that definitely was the case...

Until 1.76.0 it seems. If you use this enum on a recent version then you'll find MyEnum is no bigger than String:

[src\main.rs:11:5] std::mem::size_of::<String>() = 24
[src\main.rs:12:5] std::mem::size_of::<MyEnum>() = 24

This is because nowadays, the capacity for a Vec isn't just a bare usize but rather a Cap type that is guaranteed to be less than isize::MAX. Which gives a much larger number of representations for the compiler to use. It is defined:

type Cap = core::num::niche_types::UsizeNoHighBit;

I didn't know about this, and since Google's AI response was adamant that it only used the null-pointer niche, I figured I'd advertise it to try to set the record straight.

Let me know if this is old news to any others. It has apparently been in the wild for a couple years, but I looked and didn't see it mentioned in the release notes for 1.76.0, and I can't find a blog post that mentioned this (though I didn't look that hard). Closest I found was this StackOverflow post that gives the Cap type a mention.