I am wiring a fan with a white/black/green wires. The cable going to the switch box to fan has an extra red wire that was twisted with the black. I capped the red and connected the matched colors to the fan. At the switch box there is a light switch that is for the vanity light and one that controlled the old fan. The new fan switch has two black wires (one for fan and one for light) and a green ground wire. It is a 5 setting dial type switch. I tried a couple of things but couldn't get it to work. The box has several white wires twisted together, red and black together. Black wires together and a couple of black jumpers that were connected to the old switch I removed. What am I doing wrong? Should I re-connect the black and red at the fan and do the same for the switch or use just the Reds???
At the switch box there is a light switch that is for the vanity light and one that controlled the old fan.
So you are removing this switch and replacing it with the new one? It isn't exactly clear what you are trying to do. A drawing or pictures would help.
If the new fan has a light in it then you need to use the either the black wire for the fan or the light and the red for the other. The new switch should be fed with a "hot" wire (one that is energized all the time) and one of blacks goes to the light and the other to the fan.
Hope that helps.
White wire twisted together are very normal. that's the neutral, or return power for all the circuits.
Black and red twisted together is usually for a second hot line into a device. this is common for ceiling fans if you want to have the light on a wall switch, but control the fan by the chain. it's also used for switch controlled outlets and 3 way switches.
Generally, you don't want to change anything in a switch box unless you change how things are operating. The two black jumpers in the switch box should be the only thing you need to touch. I suspect the red wire goes into a junction box and then gets sent as a black wire for the fan.
Set the wires in the same configuration that you found them, with the old switch. Get that working again. Then use only the jumpers to hook up your switch. There was probably a double stacked switch in the box using the black and red to control separate things.