I tried hammering the screw a bit to loosen it. Any suggestion? I'm thinking of getting something to saw off the holder and hammer the screw into the wall.
The problem is most likely that the hollow-wall anchor the screw is threaded into has let go of the surrounding material. It's spinning freely. You want to either hold it while you turn the screw, or simply grind off the screw head.
Try reaching behind the bracket with a needle-nose pliers. Grasp the anchor and hold it securely while you turn the screw. Since the wall is already fairly damaged, don't be afraid to dig the pliers into the surface to get a better bite.
Otherwise, use a rotary tool or angle grinder to remove the screw head. Once it's off and the bracket is removed, simply drive the screw shank below the surface of the wall and make your repairs.