There’s a couple of problems here.
One is that the spigot pipe was not properly anchored to the wall in the first place. That renders any effort at caulking, well, kind of pointless really.
What to do? First, pull the spigot off. It either screws on to a threaded 1/2" nipple, or slips on to an unthreaded copper pipe retained by a set-screw. Likely yours is threaded.
Feel like fixing the moving spigot? Here’s a couple of ideas.
Blocking Option 1
If there's enough room to get to it via the hole in the surround, install a spacer between the outlet down-pipe and the tub surround wall. The spigot fitting should go into a 90-degree that comes down from the shower valve. Install the spacer between that fitting and the outside (wall-facing) of the surround to prevent pulling the spigot out.
For the spacer, my thought is to take a piece of 3/4" PVC, cut it lengthwise, slip it over the pipe, then wedge it so that it expands and catches the tub surround hole.
That said, it’s kind of a kludge. It won’t really work that well if somebody’s in the habit of resting a leg on the spigot for example.
Blocking Option 2
If you have access to the opposite wall, install proper blocking from the back side to anchor the spigot pipe in place. Might be worth chopping a bit of drywall to do that.
Cleaning the caulk
Whatever you decide, while you still have the spigot off you have much easier access to clean the caulk off completely without damaging stuff.
Your tub surround appears to be fiberglass. This needs some special care. Fiberglass has a gelcoat surface that can be scratched easily with scraping, or damaged with harsh solvents. Instead, use a bit of heat from a hair dryer to soften the caulk, and scrape gently with a wood or plastic tool.
More about caulk removal from Loctite: https://www.loctiteproducts.com/en/know-how/build-things/remove-caulk.html
Some other suggestions here: https://plumbingways.com/how-to-remove-silicone-caulk-from-fiberglass/
Another caulk removal idea: Borrow or invest in a steam cleaner. Use that to heat the caulk to remove it. If you buy one, besides removing caulk it works great on mildew and scale without using chemicals.
On the spigot, you can use a commercial caulk remover, acetone or lacquer thinner (use in a well-ventilated area of course.) Again, scraping with a metal tool isn't a good idea. If you want to preserve the spigot finish, use a wood or plastic tool. Again, heat works too.
Or, you could just toss that the old, crusty spigot and get a new one.
And here’s the payoff for removing the spigot
While you're at your favorite home center or plumbing supply getting a new spigot (and teflon tape if needed), pick up a spigot seal ring. This gizmo slips over the pipe and seals up the gap behind the spigot. Then if the pipe pulls out again, the seal will prevent water from getting in behind the surround.
And, bonus: with the seal doing the job, you don't even need to caulk the spigot at all.