(Ab)using a server library as a GUI - bad idea or only sort of bad idea?

Let's face it - making a cross-platform UI is hard. So hard in fact, that a common solution has become packaging an entire Chromium with your app so that your app can be a webpage -- because making a webpage is easy. So easy in fact, that even I can figure out how to make one look not terrible.

Now, I while I do like making webpages, the idea of forcing users to install an entire browser (and Chromium at that!) does not sit well with me. Luckily, there is kind of a solution: most people already have a browser capable of rendering webpages, and Rust is an excellent language for writing web servers - why not make a localhosted webpage that runs in your normal browser for the UI, then passes any actual computation or other work off to your Rust backend? Basically, a web app that exists entirely on your computer.

I'm a recent convert to Rust, so please go easy on me if this is a stupid idea, but it's an idea that I've been toying with for a while. More experimental than anything I ever plan on publishing, anyway.