Ground settles over time, and asphalt isn't as rigid as it looks. During the summer, asphalt softens a bit. It will somewhat conform to the ground under it. Vehicles will create depressions in the surface. It will develop cracks. Water gets into the cracks and softens the ground under it, making it more susceptible to deforming under the weight of vehicles.
The "holes" in your driveway look like it was previously resurfaced (or the original job was too low and a thin layer was added on top), and the added layer is what has been "erased". When roads are resurfaced, a pattern is first ground into it to create a lot more surface area and texture for new material to grab. Then a thick layer of asphalt is added on top.
If you don't prepare the old surface like that, the new asphalt doesn't bond well with the old, smooth surface. Especially when the new surface develops cracks, small pieces get knocked out, making it easier for adjacent pieces to get knocked out. And it looks like the resurfacing on your driveway was "one stone thick", so there isn't much holding that layer together.
As asphalt driveways go, your pictures don't actually look that structurally bad to me. Asphalt driveways degrade over time, but people don't usually rip them out and redo them unless there's signs of real "structural issues". If it were me, I wouldn't rip it out, at least not yet. It looks like it could be repaired and improved to get a number of years more use from it before replacement.
Keep in mind that the previous resurfacing job is falling apart. You can slow that down, but nothing you do will last a really long time. However, as long as the driveway is basically structurally sound, the old resurfacing layer flaking off is more cosmetic than structural.
Patch the "holes". You will get longer lasting results if you prepare the area first, but even just cleaning it, putting in asphalt patch, and rolling it should last several years.
The pictures don't show any, but if there are any big, deep cracks (over about 1/4" wide), fill those with a liquid asphalt repair material (unless the cracks are wider than what the product says it can be used for).
Seal the surface with asphalt sealer. This will fill and seal the spider web cracking, and add some stability to the surface.
You can get the materials at a hardware store and DIY, but it is messy. The tools and materials are heavy. A professional service can do all of this, but make sure they're reputable. A lot of fly-by-night companies just paint the surface with thinned black material that doesn't really solve anything. This won't give you the life of a new driveway, but you should be able to extend the life of the current one by a good few years at relatively low cost.
