Is it okay to let some errors panic?

I think the question makes it obvious I'm quite new to software engineering, so pardon me if the question is stupid.

I've observed that it's generally encouraged to handle all the errors using match. For past 2 hours, I've been obsessively searching all the functions that return a result so I can use match on them. However, I feel as though that there are some errors that are never going to happen under normal circumstances, and even if they do, I don't really know how I will keep the programing running. I want the program to crash in such extremely rare cases, is this an acceptable approach?