We have a dumpster on our property for another home project and are hoping to take advantage of this by disposing of some bathroom lights and a ceiling fan. However, we do not currently have a replacement for these items.
I don't believe turning the breaker off (and leaving off) is an option.
If I were to remove these lights and then cap off with wire nuts and electrical tape would that be a safe option or is this just asking for trouble?
With the breaker off, put a wire nut on each wire separately (don't nut them all together!) and you're good to go. That is the proper way to terminate unused wires, whether the breaker is to be turned back on or not.
In addition to just capping with wirenuts, depending on your wiring configuration and your expertise, I would suggest labeling the wires as you remove them (maybe if you use large enough wirenuts to write on put a different number/letter on each one and keep a log of what each letter's wirenut was connected to) so you'll know what was connected where in each junction box when you eventually install the new fixtures. This will avoid a future SE question on "how do I know what should be connected to what" question that's all too common.
The simplest way to make the bare ends of the wire safe at the device end once the device is removed is to remove them from the breaker at the supply end of the wire.
This is common practice in industry where something will be disconnected for medium to long term. Isolation procedures and leaving things locked and tagged off are another way to do it, but removing the wires from the supply, taping and labelling them and leaving them in the bottom of the box/cabinet is usually preferred. A variation is to then connect the wires to an earth terminal/connector so that when working on the remote end one can simply verify via a continuity test that the wires are all common to earth and thus incapable of preventing a danger.