Your best bet is a scraper and sander. I wouldn't go near fascia with a power washer for a few reasons... one being the pressure could upset your balance while standing on the ladder. Forcing water into wood is never a good idea. You can shred the fascia (it's usually pine) for one.
First, I'd wash the existing paint with TSP to get the chalk off it. Most people don't know this, but paint chalks to "self clean". This step takes the dirt and other environmental fallout off the surface. Then I'd scrape and sand it. Fill in any irregularities with exterior spackle (makes the wood look smooth and disguises the unevenness of the surface), feather sand the spackle, and then wipe all of it down with a damp cloth as the dirt will interfere with the surface adhesion. Use a good primer - either latex or oil - let dry and finish.
Another alternative is get a "painteater" and run the entire surface - and remove the paint down to raw wood.