Go ahead and use a variac (variable transformer) to slow down a box fan to a reasonable speed. Just make sure it's moving enough air to cool its own motor.
jwh20 isn't wrong, but his explanation doesn't apply to your use case. He's right about not using voltage as a speed control for a motor with a fixed load. Since you don't need a certain amount of power to move air, it's not applicable here. (Using a variable frequency drive is the preferred way.)
The difference is that your box fan motor isn't connected to a fixed load. Air resistance increases as fan speed increases, and at low speeds the fan has less resistance.
I just tested it with two different fans on my variac and a Kill a Watt meter. (It's possible these aren't the same as a box fan though, both of my fans have run capacitors. I'm not sure if box fans have run capacitors or if that's the same as a shaded pole AC motor.)
As I lower the voltage, it also lowers the amperage it draws. The only problem would be if you lowered the fan speed too slow. A fan motor drawing 10 watts that isn't moving any air is going to get hot eventually.
Just keep it above 40% (about 50 VAC) and you'll definitely be fine. One of my fans takes 28 watts at 50 VAC, and 112 watts at 120 VAC. When it's running slow, it's moving fast enough to cool itself fine.
I'll leave it running at 15 watts for a while and let you know how hot it gets in an hour or so. It's barely audible at this slow of a speed, but it's too low of a voltage to measure directly with my Kill a Watt (and I'd rather not stick my multimeter in the socket just for the data.)
Edit: At 15 watts and an hour or so, the fan is 15 degrees F over ambient.