When you bring a crawlspace into the building envelope by insulating it like this, you need to treat it like any other room and condition it.
Putting a fan in the wall that exhausts to the exterior defeats the whole point by circulating outside air through the crawlspace--which is now a part of the interior of the house. There's no point in insulating a space that has free air movement between the inside and the outside. The only reason why the fan makes a difference is because it's moving air, which is good. But it's also bringing in outside air which in the summer is humid; you're replacing very humid air with slightly-less-humid air. That's why the problem hasn't completely gone away. If you wouldn't open the windows to dehumidify the rest of your house, you shouldn't do it to your now inside-the-building-envelope crawlspace. Think of your crawlspace as now a room, not a crawlspace. Maybe it's a crawlroom.
Put HVAC supply and return vents in the crawlroom and the problem will disappear. During the summer, your AC unit will circulate air through it just like the fan is currently doing, but also dehumidify it, which is what you're currently missing. This will be much cheaper than running a separate dehumidifier. The very last thing you want to do is run a dehumidifier in a room with a ventilation fan like the one you have rigged up: you'd be trying to dehumidify the whole neighborhood.
Edit: if the floor doesn't have a vapor barrier of some sort, that also needs to be addressed. When you bring a crawlspace into the building envelope, you can't do a half job and forget about the floor. If the floor slab doesn't have insulation or a vapor barrier underneath it, then you need to add one on top of it, and seal it to the foamed walls with more foam.