That looks like a fairly old style of phone jack (before punch-down jacks became common). That style of jack took up an entire 1-gang box. Installations like you have in your photo are typically someone's attempt to expand the system without needing to bust open the wall to run more wires or install a larger box.
If you take off the faceplate you'll likely see that both the phone line inside the wall and the protruding wire are connected to the back of the jack. If this is the case, you should be able to easily disconnect the protruding wire without altering the jack's function.
If you find that the protruding wire is not connected to the jack, then it might be a second phone line that was fished through the same wall opening and then run directly to a device in another location. I've seen this in several instances where a second line was added after the fact. If this is the case, then I recommend replacing your existing jack with a modern keystone-style jack which will let you fit two jacks into the existing 1-gang box. Connect the line from the existing jack to one of the new jacks, then cut the protruding cable inside the box and connect it to the other jack. That will eliminate the annoying cables while making it easy to connect to either line in the future if desired.
Even if you never plan on getting land-line phone service, I encourage you to avoid doing anything that might prevent this line from working. You never know when you'll change your mind, and re-running cables inside the walls is a pain. Some alarm systems require a phone line, which is a common use case for the "hard-wired cable running into another room" scenario. You can still call 911 on inactive phone lines in some locales, so keeping the wiring intact (plus a simple, corded telephone in the closet) might save your life in an emergency.