Priming those small areas could be a factor, but not likely the sole or primary causal factor of an undesired issue with the final surface treatment appearance. The difference in the surface texture at the repair sites will be your nemesis.
The way the light reflects from the surface (look up hold-out factor) will change the way the repair areas look compared to the previously painted areas. In critical applications, professional decorators would skim coat the entire wall with plaster to avoid this issue. All is not lost, however. Flat and low-sheen paints are much more forgiving than glossy paints. Walls where the light strikes perpendicular rather than at an angle makes imperfections hard to see. Use of a paint roller with a bit of nap will really help hide surface imperfections.
And there is always texture. You could use a plaster-base texture material to completely hide the repair, or use a textured paint additive.
In the end, if this is a normal home with normal people looking at it (as opposed to a museum, showroom, executive suite, etc.) you will probably be the only one who sees it.