At every faucet and toilet in our house, when water is running, it creates a loud wooshing noise in the walls. I'm not sure how else to describe it, but it's like the normal "oh I can hear the sink or toilet on upstairs" but VERY loud.
What, if anything, can I do to reduce or completely eliminate the noise?
Make sure the shutoff valves at your toilet and sink are fully open. Inadvertently leaving them partially closed can cause sounds like you describe.
EDIT: Just noticed you said "at every faucet and toilet in our house." I'd still check all your shutoffs just to be sure, but also make sure you check the main shutoff valve for the whole house.
A disturbed water flow causes such sounds. Since the sound propagates through the entire plumbing, the problem may be hard to pinpoint but is generally located at a valve or an endpoint.
Check that all valves are opened completely (and when applicable, close them half a turn to prevent them from getting stuck over time)
Try removing one of the faucets and letting the water flow freely; is the noise still present? If not, then your faucets may be in need of descaling or replacement.
Try to determine the shared stretches of pipe the water is flowing through for every 'waterpath' on which the noise is present. Examine those stretches first.
As you can see from this question opening water pipe tap has hissing noise throughout the house. Noises can also be caused by over pressurization of the plumbing system.
To determine if the plumbing has too much pressure, you can pick up a water pressure gauge from the local hardware store for ~$5.00 - $10.00. They are designed to attach to an standard hose bib, and can be used to measure static water pressure.
If you find that the system has higher than normal pressure (between 40-65 psi is normal), then you should first make a call to your local municipality and ask what the static pressure outside your house is. If the pressure outside the house is as high as the pressure you are reading inside, then you will have to install, adjust, or repair a pressure reducing valve inside the house on the main service line (typically after the meter).
If you already have a pressure reducing valve, you can try to adjust it. If that does not work it likely means the valve is bad, and will have to be replaced. If you don't have a pressure reducing valve or the current one was found to be bad, you'll likely want to call a plumber to install/repair the valve. If you're handy enough to fix/install it yourself (with proper permits of course), then you know what to do... so get too it.
Some form of insulation, either normal pipe cladding or expanding foam could be used to reduce the noise.
In addition to the main shutoff valve, I would also check any other valves or devices at or near the main shutoff, e.g. a back flow preventer or a pressure regulator valve:
