For short-term loads, the easiest solution is a UPS.
Solar has some problems as a backup power source. It's expensive (panels + controller + inverter + transfer switch + wiring + installation). Chances are your home is not an ideal location to collect solar (shadows, latitude, roof orientation). The sun is often unavailable just when you need it (winter storms).
On the other hand, a UPS will charge from the wall, produces a very high quality output, is reasonably priced, and is trivial to install.
Regardless of which technology you choose, there are two important measurements: maximum load (how much power you can draw at any given time) and capacity (how long you can run at a given load). A simple network router is a very small load, max load won't be a problem.
For occasional use, one option is to use your car's 12V system to power your router. You can get a cheap adapter like this one. Idling your car's engine for just this purpose would be terribly wasteful, but if you do it rarely it would be OK.
As a very simple hack, you may be able to just connect a regular 9v battery to your router.