Pencil marks will not have much effect on paint adhesion, and will not cause stainiong, but they may show through light-coloured paints.
So yeah either do a test to see if they show through, or just erase them.
That looks like regular dried-up old glazing putty. If it was me I would soften it using a heat gun while scraping it with a putty knife or similar tool.
Then I would rig a guide on my router and remove all remaining putty (and maybe a thin bit of the wood) with the router. I might use a paintbrush to soak it with mineral spirits prior to routing, to reduce ...