Polymeric sand is basically 85% to 90% sand and some sort of cement (either portland or organic). It's going to complicate the repair job due to it's binding abilities, but it certainly won't make repair impossible.
Some types of relatively easy fixes become a major pain when the sand is glued to your pavers. Pulling broken bricks is harder when the sand is gluing them in. Correcting low compaction becomes more expensive. Re-installs and touch-ups often require removal of the old polymeric sand (according to manufacturer's recommendation) which is also complicated by the binding agent.
Cleanup is more expensive. If not done on a very dry day, stray sand tends to glue in place. That means that on install, your pavers should be spotless, as well as your surrounding walls, doors, frames, etc. Otherwise, you're scrubbing off sand / cement instead of just sand.
It's still a good product, but perhaps some of these reasons might give you an insight as to why your contractor sees it as a mixed bag.