It would be best to remove the paneling. The plywood paneling is probably 1/4" thick. You can buy drywall @ 3/8" thick, so not a huge change, just an extra 1/4". In addition to your door jambs, you probably have electrical outlets to account for. Unless they have exceptionally long screws, you'll run into some trouble there.
Removing the paneling isn't really hard work, or even too long a task. As suggested in the comments, it might be helpful to see what is behind that paneling so you know where to be careful. It might be quite insightful. At a time like this, you also have the opportunity to lay in any structured cabling you may want (cat6, cable, etc).
The paneling may be behind your door jamb casing (the part on the same plane as the wall). I've seen it both ways. You'll know pretty quick once you start removing a panel next to a door. The casing is removed by cutting both sides with a razor knife (cuts free paint and caulk), then using a small pry bar or flat head screw driver to move the casing away from the wall. It breaks easily, so careful and listen for cracking sounds. It may be required to shim out the casing when you reattach it. You can cut strips from the paneling to do this.
Just a thought, and it may not apply to you, but if you're hesitant about removing the paneling based on time, the time it takes to remove the paneling is nothing, absolutely nothingm compared to the time to properly do those drywall joints (oh, and the mess!)
As for the seam where the brick is, some shoe molding should work. Based on the picture, it will be fine at the ceiling, but I didn't see a picture of your baseboards. If you're experienced with a coping saw, you should be OK with the baseboards too.