What's so scary about `mem::transmute`?

54 points giantenemycrabthing

tl;dr: Under which circumstances would the use of mem::transmute be tempting yet unwarranted?


Motivating example

I recently had to write a library about an enum that had 48 different possible values¹. There were numerous functions that took one or two such values, performed some bit-wise operations, then output a new such value. It was meant for microprocessor use, so it was very important to be panic-free, plus it also had to work in const contexts.

Now, getting the underlying u8 out of the enum is very simple: A simple as does the trick. But… what about the reverse? Taking a u8 and making it an enum?

Answer: core::mem::transmute. Lots and lots of core::mem::transmute.

So, how can I know that this is sound? I have no fundamental guarantees that there's no UB… but what I do have is miri, with which I've checked each of my functions exhaustively.

¹ 3D rotations that preserve octahedral symmetry, if it matters.

So, when should one not use mem::transmute?

I'm fairly certain that, in this specific example, the use of transmute is both warranted and correct. Other examples I can think of is to_ne_bytes from the standard library, which is merely a one-liner transmute under the hood.

However, I've noticed some extreme apprehension surrounding this function from well-respected Rustaceans. Ralf Jung has said “If using transmute—just don't, I hope!”. Cliff Biffle has said “transmute is like a reciprocating saw: it’s always the wrong tool for the job, except when it isn’t”.

So: Personally, in all the examples I can think of where the usage of transmute is tempting, it's also warranted. However, I have to assume that there are examples where it's tempting but unwarranted, or else there wouldn't be so much fuss about it.

So: Could anyone mention such examples? I feel as if I know what to do, but not what not to do, and knowing both would make me feel safer.


Edited to address some of the comments here: My question was actually about transmute in general, not about this specific motivating example. But, since it garnered so much interest…

  • Yes, my enum is [repr(u8)].
  • Yes, I'm implementing TryFrom. (Using, you guessed it, transmute.) But I usually can't use it, because of…
  • An important thing that I forgot to mention: The enum ↔ u8 casts need to also work in const contexts. TryFrom is no help here.
  • I do actually debug_assert that the value is within bounds before transmuteing. It's only when debugging because in my case errors are considered a fault of the library, not of the user. (The library models mathematical operations, and as such has no business panicking or erring.)
  • I used to use num_enum. After a bit of refactoring, it ended up only saving me 8 lines of code, so I removed it. As for bytemuck, I didn't know about it and so I didn't try it.
  • The primary reason to flirt so much with unsafe is not so much performance as much as panic safety. Performance is a very close secondary reason.
  • I did actually debate using a helper function with a gigantic match expression. I ultimately decided against it, because a) it would be much less readable and b) it wouldn't actually solve any problems. (Again: Any attempt to transmute an invalid value would indicate a fault in the program.)
  • For whomever might be curious, I published a simplified version of some of the operations.