It's because the dimmer is a throwback to the 1960s.
A lot of switch wiring is installed as switch loops - only 2 wires come down from lamp to switch. The intention is that a plain switch will either connect (short) those, or not connect them. The two wires are always-hot, and switched-hot (i.e. hot when you want the lamp to be on).
Then they figured out how to use silicon triacs to dim incandescent bulbs cheaply and without making a lot of heat at at the switch.
Early dimmers had a problem. The dimmer's onboard electronics needed power - easy enough; grab always-hot and neutral from the box. But neutral wasn't available in switch loops. So the dominant dimmer design leaks power through the incandescent bulb in order to power the dimmer's electronics. Even in the nominal "off" position, this leakage still occurs.
On incandescents, this was not enough power to make the filaments glow cherry red even. So it was unseen.
With LEDs, we should have done a blank-sheet design where LED dimmers send full power, but also digital pulses telling each LED how dim to be, and the LED would decode that and set brightness to match. Sadly, they stayed with triac dimming, aiming to be backward compatible with incandescent bulbs.
So we are stuck with the many absurdities of triac dimming. This "leakage current" is one of them.
Obviously your fixtures have multiple bulbs, and one "diagnostic technique" we advise is to replace ONE of your LEDs with an incandescent bulb. If the problem suddenly goes away and the remaining LEDs behave properly, we know we are dealing with this problem.
Lutron, a dimmer maker, threw up their hands and came up with a gadget called the "LUT-MLC", which does the same thing as that 1 incandescent, giving a bypass path for that leakage current. It is wired in parallel with the existing bulbs.
Some makers of "dimmable" LEDs put similar electronics in their LED "bulbs". Others do not, and expect you to use a better dimmer (e.g. that uses neutral).
To support smart switches and better dimmers, NEC 2011 now requires switch loops bring down a neutral wire.