What you are seeing is the effect of Stray Voltage caused by capacitive coupling. Electricians refer to this sometimes as Phantom Voltage.
Ungrounded metal objects close to electric field sources such as neon signs or conductors carrying alternating currents can have measurable voltage levels caused by capacitive coupling. Since voltages detected by high-impedance instruments disappear or become greatly reduced when a low impedance is substituted, the effect is sometimes called phantom voltage (or ghost voltage). The term is often used by electricians, and might be seen, for example, when measuring the voltage at a lighting fixture after removing the bulb. It is not unusual to measure phantom voltages of 50–90 volts when testing the wiring of ordinary 120 V circuits with a high-impedance instrument.
Sections bolded by me
The voltage you read with a common multimeter can change depending on how the wire is run, how long it is, etc, but you will usually be able to measure some voltage even when a light switch is turned off. If you connect even a tiny resistor between the phantom hot and neutral, the voltage will disappear. It is very, very low amperage and can't be felt or do any work.
Very simple LED bulbs that have the bare minimum of driver components can be illuminated by this voltage because they are very low power devices. LED bulbs that look more like regular light bulbs from major manufacturers have complex circuits to drive them and they typically will not show any glow.