In a rim joist it should be no problem but I would make it a tight fit, check the size of your outlet it may be 3” so the pipe will slide over it.
I like the hole to be as small as possible I know the dryer vents I have installed for 4” pipe worked with a 4” hole.
I don’t care for the corrugated or flex pipe unless there are several bends and even then only for a retrofit. The sticks of 3” galvanized are only a few bucks but this is my opinion some don’t care about the stuff that builds up in the pipes but it can be nasty the straight sides don’t tend to collect moisture.
In my state we have to have a timer on bathroom fans to run after the lights are turned off this helps to dry out even corrugated pipe.