Most Likely Not OK
###Most Likely Not OK YouYou probably have neutrals mixed up. The one exception is an MWBC (multi-wire branch circuit). I doubt that is the case based on your description), but that is a possibility. Assuming that is not the case:
In addition to the (obvious) hazard when working on wiring, there is another problem. In normal US residential service, neutrals are not protected from an overcurrent condition. Normally this is not a problem as the current on neutral = the current on hot, and hot is protected. However, with two circuits sharing a neutral, there is no guarantee that the neutral currents will equal the hot currents, and this can be very dangerous.
For example, if you have two 20A circuits and each one has a 16A load and the neutral fails on one of them then you will now have 32A on one neutral wire, which is more than the standard 20A circuit wiring is designed to handle. The hots will still have 16A each, so the breakers won't trip. But the neutral wire with 32A will burn up.
In order to fix this, you need to find out where that neutral wire comes from and what hot wires it is paired with. Even though all the switches were off, you may have a hot for another circuit passing through this box that is on the other breaker. If that's the case, it just needs to have its neutral separated from the others. But you may find nothing from the other breaker (except this neutral) in which case you have to look elsewhere, starting with any junction boxes for anything (e.g., lights, receptacles) that is powered by the other breaker and see if you can find something that is connecting back to this box that shouldn't be there.