You've made a circle. That's not allowed in AC power, and for good reason.
The white wire is spare from SW1 - SW2 - L2. You need to use that for neutral instead of having the white shortcut from L1 to SW1. Yes, that means the white "doubles back" physically and travels 3 sides of a square.
The operating principle is Currents should be equal and opposite in each cable or conduit. No loops; tree topology only. If hot goes up a branch, neutral must come back that branch.
In other words wiring should be (if straightened out)
\ | /
b = Always-hot
w = True Neutral (must be white)
y = Travelers (will be black or red; remark them yellow)
r = Switched-hot (will be black; remark red)
Fix that first because it's impossible to even diagnose the other problem while this problem is so broken.
If you have been wiring with individual wires somehow, stop doing that. Individual wires aren't listed for direct use.
If you follow my color advice, things get a whole lot easier, especially if 3-way circuits are involved, which (without wire remarking) are confusing on a good day.