A lump hammer should do the job.

If that doesn't knock the bricks off you might need to use a cold chisel as well. You want to choose a wide bladed one and attack the mortar.