It sounds like you have light bulbs or fixtures directly controlled by Alexa/Google Home. A better solution, though sometimes a bit more work to install, is a smart switch. With a smart switch, the actual switch is replaced with a device that functions as both a physical switch and as a WiFi/remote/voice-activated/etc. device.
There is actually a bigger problem, potentially, than simply inconvenience. With your current setup, if the lights are turned off via Alexa/Google Home and then there is an emergency and someone who doesn't know about your Alexa/Google Home setup needs to turn on the lights, can they do it? Maybe (depends on the device) turning the switch on/off multiple times will do it. Maybe not. This is a real safety concern.
Related: you also need to ensure that you can toggle the lights regardless of whether your home assistant is available. If you can't turn your lights on because your Alexa is updating the firmware or because a backhoe tore through your fiber cable, that is an equally real safety concern.
A proper smart switch can often work in a 3-way configuration - i.e., with the 2nd 3-way switch being either a regular 3-way switch (possibly rewired a little bit) or with a "remote" at the other location (possibly wired using the existing wiring, possibly battery powered and radio controlled). There are a lot of possibilities.
Back to the original problem. There are lighted switches available such as:
However, as with smart switches, dimmers, timers, motion detectors, etc., lighted switches have to get their power from somewhere. There are 3 options:
- Neutral - Arguably the best, but neutral is often not available in switch boxes in older homes
- Ground - Usable in limited circumstances. Do not use ground instead of neutral unless the device is designed to do so.
- Leak current through the main circuit - Lighted switches used to all work this way. Many still do. The problem is that this is generally not compatible with LED lighting.
If you decide to install lighted switches, make sure they are compatible with your wiring and with your current and anticipated lighting fixtures.