We have old all-aluminum windows, and they've worked well after decades. But they're just a tad bit dirty.
Are there any cleaners that will help me clean these windows?
If you just want to give them a light cleaning you can use a few pads of 0000 steel wool pads and a little elbow grease then spray with clean water. If you have aluminum discoloration or mineral deposits you may want to use a chemical such as Aluma-Kleen (which is bio-degradable) or a even a homemade recipe my grandmother use to use was 1 part water to one part vinegar (acid) to clean aluminum. Always rinse with lots of clean water and wipe down dry to prevent further minerals from staining. Hope this helped.
Try ultradilute hydrogen peroxide. Typical strength here in India off the shelf is 4% sold in volume of 200ml. I usually dilute it 10x;/in 2 litres of water. It works reasonably but also demands elbow grease in ample amount.