This is a code violation. But there are real specific reasons (i.e., not just to have all switches look the same). A specific why this is bad:
With the neutral switched instead of hot, if you turn off the switch to change a light bulb, you still have a hot socket. Since ground is present (neutral is effectively ignored since that has been switched off), you could still have a complete circuit (e.g., if a bulb breaks) where normally you would not expect it. Actual maintenance work (beyond changing a light bulb) should be done with the breaker off, but most people would not bother just to change a light bulb.
And even if you remember this, the next owner might not know about this little "quirk". Which could be a shocking experience.