Vinyl siding (along with steel) is fastened to your walls with nails that aren't fully set. This allows lateral movement when the siding expands and contracts due to temperature change.
Ideally, the nails are set just snug enough to eliminate in-out movement, which is probably what you're hearing. If they were left out too far, some movement can occur. In fact, since vinyl is very flexible, some movement (slapping) could occur even if the nails are snug. This is one of the drawbacks of the "maintenance free" era.
Unfortunately, there isn't an easy fix. Some underlayment sheeting is embossed or rippled, which can help, as can insulation panels. That's a before-install solution, though. At this point you'd have to loosen each panel on the wall by unclasping the bottom edge, drive the nails a little further, and re-fasten the panels. With the right tool it's not difficult, but it would take a while.
Do make sure that it's actually the siding making noise, however. Check rain gutters and downspouts, window shutters, etc. where applicable.