If the hole appears to have been formed from the inside out, it is possible that it is caused by rusting of the metal. The orange powder may be dislodged rust, which would confirm the above supposition.
Compressors fill a tank with outside air, causing it to become warm. As the compressed air cools, water previously suspended in the air as water vapor will condense and collect on the bottom of the tank. Many larger tanks will have drain valves to allow removal of this water.
Smaller tanks frequently do not have these valves and can be drained by turning the tank in such a manner as to position the exit hose opening at the bottom. Expect a spray of water to appear when opening the air valve.
Unfortunately, if you are experiencing enough rust to create the hole you've noted, the rest of the tank, or a good portion of it is also rusted.
As an extended process, one could pour a quantity of rust treatment product into the tank, then rinse out the resulting sludge and debris and allow the rust treatment product to dry. Following that, a sealant of the type used to seal fuel tanks could be used. This would prevent additional rust from forming and very likely close the small hole you've identified.
I understand that the use of fuel tank sealant is unpleasant. In the case of a high pressure vessel, such methods are not recommended, see Isherwood's answer as reference.
All of the above also requires that you have a suitably sized opening to access the inside of the tank. Few small compressor designs will have such an opening.
It may be necessary to retire this device.