This rather depends on where you are, and therefore what kind of switch you have.
Some jurisdictions' switches are completely encased in plastic, with no metal - not even screws - showing on the outside. There is no way for house power to get out of the device if a wire comes loose and touches something it should not be touching.
USA switches (likely others as well) have metal screws holding the switchplate covers on. These screws attach to the metal mounting strap of the switch body. If a loose wire touches that strap, house power can get to the person touching the switchplate screw and give them a dangerous shock.
Effort should be made to ground any device being wired in that has metal on its outside.
Additionally, if the manufacturer put a screw or wire for ground on the device, then there is a foreseen possibility that there is a risk of electric leakage through the device. They don't put in extra parts that drive up manufacturing costs that aren't needed.
TL;DR: If there is a ground wire or ground screw on the device, ground it. There is a reason the ground point is there.