Why does Rust standard library use "wrapping" math functions instead of non-wrapping ones for pointer arithmetic?

When I read std source code that does math on pointers (e.g. calculates byte offsets), I usually see wrapping_add and wrapping_sub functions instead of non-wrapping ones. I (hopefully) understand what "wrapped" and non-wrapped methods can and can't do both in debug and release, what I don't understand is why are we wrapping when doing pointer arithmetics? Shouldn't we be concerned if we manage to overflow a usize value when calculating addresses?

Upd.: compiling is hard man, I'm giving up on trying to understand that