My question is: How do I stabilize this crack?
Can I fill something into the hole so that the crack doesn't get any worse?
The rule with splits or breaks like this is
Don't skip #2 or you'll have wasted the glue. Clamping forces the glue into the wood, and it will be a stronger bond when cured, than the wood alone would be.
I would remove the screw, glue, clamp, and then take a drill bit the size of your screw head and chamfer the hole (just rev the drill and press lightly to let the drill cut a bevel where the head will sit). The chamfer should ensure it doesn't crack when added again.