This is all LED related.
Option 1: You are changing from incandescent to LEDs.
The new LEDs you are using do not play well with the dimmer, smart switch or other smart electronics currently installed. Those old electronics depend on a unique characteristic of incandescent bulbs, which most LEDs do not bother emulating. When you change the last incandescent to LED, that breaks the smart device. The fix here is to address the incompatibility (change one or the other) or add a Lutron LUT-MLC, which provides the characteristic the old dimmer/etc. is looking for.
Option 2: You are using incandescents, not LEDs.
Most fans have a safety circuit designed to prevent you from overlamping the light sockets, meaning put in bulbs that are too large and make too much heat. These would damage the fixture or start a fire.
Adding the fourth bulb pushes the power draw over the limit, resulting in the safety circuit doing its job.
The cure here is to use LED lights, which draw vastly less power, and can give plenty of light for far less heat. The fixture cares about the heat, not the light.