Building one binary that targets multiple CPUs?

My program has up to a 30% performance increase when I compile for Haswell with -C target-cpu=haswell instead of the default x86_64 target CPU. I'd like to build a portable binary, but have it benefit from the performance increase when run on compatible CPUs.

The obvious solution is to build two binaries separately. The portable one would run first, and if it detects that the CPU supports Haswell features, then run the optimized one instead. The two binaries can be combined with something like makeself. But, this feels like a kludge.

Is there a better way?