If you're in the coastal area, the attic air during the summer will be hot and humid, so it won't be nearly as effective at picking up moisture in the drier. Outside air won't be much better.
The drier doesn't operate many hours per week, so in the long run, you aren't wasting much conditioned interior air; plus the conditioned air will dry the clothes faster.
I wouldn't mess with trying to feed outside or attic air to the drier. Unless you add vents that seal when air is not being drawn, you will introduce additional load on your heating and AC that will affect your energy use year around.
If you live in a desert area, the outside air could be better than conditioned air for drying. However, that would be worse of an HVAC load without keeping it sealed out when the drier is not in use. I suspect that the cost of trying to use desert air for drying would far exceed the value of shorter drying time, or at best, would be such a small benefit that it wouldn't be worth the cost and effort.
A couple of solutions that don't require major work or piping in external air:
Just run the drier with the closet door open, and close the door when it is not in use. That doesn't require any changes.
If you want to be able to completely close the closet door when the drier is running, it doesn't take much gap around the perimeter of a door to provide adequate air. There is typically a decent gap at floor level. Look at your other interior room doors. If there isn't a similar gap on the closet, you may be able to raise the doors a hair, or trim the bottom edge. If the gap is similar, it will be adequate and not noticeable.