Yes, so long as you are removing a 4-way. You just need to (turn off the breaker first!!!!) tie the two inputs to the two outputs at that box -- doesn't even matter which goes to which. Use proper-sized wire nuts, and as you suggested, put a cover on the box. It is a code violation to leave the box inaccessible, so don't plaster or drywall over the cover.
If you want to remove a 3-way, I strongly recommend you replace the next 4Way in line with a 3way, leading to a 3-4-4-3 sequence. Tie the hot line into one of the output lines from the 3way you're removing, and cap off the other output line there AND as it enters the 4way location which you are changing to a 3way. Drop the 3way into that location and you're done.
(BTW, you could use the existing 4way by simply disconnecting the 'dead' input to it from the former 3way location. That may violate code, so check with a local electrician first)