This is what I'm dealing with - rudimentary efforts have not paid off and I feel like I must be missing something.
Round-Up 365 will stop frequent reoccurrence. But, with Round-Up 365 only, you spray everything down & when dead & dry in 2-weeks, just scuff the remnants away with your shoe or a string trimmer.
Birds & wind are your enemies & the only way to avoid this entirely is to go with slab concrete. Not a good option unless it's colored concrete, your pavers look worlds better. You'll need to repeat the Round-Up 365 & shoe or string trimmer annually or bi-annually.
Filling the paver joints won't do anything good. It would ruin drainage, looks & the pavers much needed movements.