I don't know what this cap is... is it ok to cut it off and splice the two yellow wires together as I have been with the previous splices.
Yes, that would be fine. What you see is one side of a push-type connector that allows the wires to be disconnected and reconnected.
These are called Bullet connectors. They're used as a male/female pair, and allow for quick, repeated connection and disconnection of a circuit. They can be found at any automotive supply store, and some hardware stores.
If you won't need to disconnect and reconnect the wires, you can simply cut the connector off, strip the wire, and use whatever other type of connector you'd like. Just make sure the connector you choose is rated to handle the size of wire, current, and voltage that it will be exposed to.
Typically wires used in car stereos are 18 AWG, though sometimes the power (Red) wire is larger. The blue butt slice connectors you're using are rated for 16-14 AWG. You should be using the red 22-18 AWG connectors instead. You may find that with the larger connectors, you have to squeeze the crimp tool really hard to make a good solid connection. The wire size should be printed on the wires.