Dig out enough to put in backer rod under the new caulking, and apply new caulking. Which is how it should have been done in the first place.
Foam backer rod is available in multiple sizes and is somewhat compressible (so if you have a 1/2" gap, you want something like 5/8" backer rod so it's firmly squeezed in place, and then it supports the caulking on top of it, rather than having a huge amount of caulking filling a huge gap.)