Yes, you can usually reuse the jacks and cabling
Most of these points have already been touched on in the comments, but here is my full answer:
1 - The jacks can be reused as long as they are in good condition. The main thing you want to watch out for are bent pins where the patch cable (aka the network wire from your computer) plugs in. There can be bent pins that don't affect the 2-wire phone cable but that will affect the 8-wire network cable.
2 - While you only need 4 wires connected for standard networking, if you ever use Power-over-Ethernet or some higher-speed connections (which admittedly may need higher quality wire) then you may need all 4 pairs. I normally (and for many years now) connect all 4 pairs routinely to future-proof things.
3 - You can basically just pull the wires off the jacks, cut off the existing end (just the last inch or 2) to have clean cable, strip more of the outer jacket if needed and then punch down.
4 - The key is to use a real punch tool such as Cable Matters 110 Punch Down Tool with 110 Blade . More $ will get you a better quality tool, but if you use it for just a few jacks then anything will work fine and will work a LOT better than trying to use a screwdriver.
5 - T568A or T568B doesn't matter. Just make sure you do the same on both ends.
6 - Your router or switch needs a separate home run to each device. Ideally you will put that where both cables terminate in one location and that way you can connect a computer at each "phone jack". Alternatively, you can put the router (and other devices connected directly to it) next to one jack and use a patch cable where the two cables are together to go from one jack to the other and put a computer at the other location. Quick ASCII art:
1 --------------------------------- 3
So either router at 1/2 connecting to both 3 & 4 or router at 3, patch 1 to 2, computer at 4.