You mentioned having a 3 way switch for the pantry. Is there actually a second switch somewhere that is also supposed to control the pantry light, or did someone just throw in a 3 way switch in a situation where a normal, single pole switch would have worked? How many screws are present on the switch that controls your fan and fan light, not counting the green grounding screw?
Where does the power enter this circuit? It could enter the box with the double light switch, the box for the pantry light switch, the box the fan is mounted on, or the box that the pantry light is mounted on.
Helping you rewire this mess to work properly is going to require detailed information about exactly what wires are present in each location. There are several "right" ways to do the wiring in this situation, and picking the RIGHT "right" way for you will depend upon what wires are already present.
I'm going to go out on a limb with one possible answer to your question. Just keep in mind that it's only a guess since I don't have all of the information needed to know for sure.
You know how a duplex receptacle has 2 sets of connections on it, and you can run power in to the top two screws and out through the bottom two screws and on to the next outlet, right? Well, it may be possible that some very inexperienced person in the past saw a 3 or 4 way light switch and thought they could do the same thing. If they tried to daisy chain the power between light switches the way you would do it between receptacles, and they chose just the right (or, in this case, WRONG) way to attempt it, it could lead to your current situation.
In any case, the situation needs to be fixed sooner rather than later. The fact that your lights are functioning this way indicates that whoever did the electrical work had absolutely no idea what he was doing, and he VERY likely created unsafe conditions in your home. If you aren't up to making a diagram of the current wiring configuration for us to look at here, then you really, really need to get an electrician in to sort out your problems before your house burns down or someone gets electrocuted.