Oh dear, some dim bulb wired this dimmer upside down and backwards
Whoever installed this dimmer clearly hasn't a clue, because they wired it in a way that's been forbidden for the past almost-a-century now! In particular, they put the dimmer in the neutral side of the circuit, instead of the hot side, where it belongs. The NEC prohibits this in section 404.2(B):
(B) Grounded Conductors. Switches or circuit breakers shall
not disconnect the grounded conductor of a circuit.
Exception: A switch or circuit breaker shall be permitted to disconnect a
grounded circuit conductor where all circuit conductors are disconnected simultaneously, or where the device is arranged so that the grounded
conductor cannot be disconnected until all the ungrounded conductors
of the circuit have been disconnected.
Furthermore, they managed to abuse the safety ground as a neutral (a working current return path) and the neutral as a half-hearted grounding attempt, giving this setup even more zap potential than it normally would have!
Getting this fixed
The first step in fixing this is to turn the breaker off, remove the errant dimmer from the circuit, and use a multimeter to check for continuity between the white wire that the dimmer's green wire was connected to and the bare wires in the circuit. If you don't have continuity there, the feed for this box will need to be reworked due to the neutral having gone bad.
If you do have both a good neutral and a good ground going back to the panel, though, we can move on with fixing this then. To get this into a proper state, we need to undo the dodgy tape-splice on the grounds and replace it with a proper wirenut, incorporating the green ground wire from the dimmer into this redone splice, then remove the wirenut from the two black hots and use a new wirenut to splice the white neutrals together. Once that's done, one of the black wires from the wall gets nutted to the black wire on the dimmer, and the other black wire on the wall gets nutted to the un-taped red wire on the dimmer. At that point, you can button things up, turn the power back on at the breaker, and enjoy your dimmer without having to worry about any shocking moments, during lightbulb changing or otherwise!