You can't run ground (or any individual wire) outside conduit in the ground - it needs to be protected from damage.
But you really don't have a problem. Based on a quick look at some conduit fill tables, you can run a minimum of 5 x 6 AWG wires in a 1" conduit (Schedule 80 PVC), and even more in other types of conduit.
You also don't need a 6 AWG ground. See NEC 250.122. Based on a 50A circuit, 10 AWG is fine.
Just realized you are using copper. Unless local rules forbid it, you are much better off with Aluminum. 50A is actually OK on 6 AWG aluminum, though you need an 8 AWG aluminum ground instead of 10 AWG copper. But you can likely save a lot of money even if you go with much larger aluminum wires, which would give you increased capacity for the future if needed. If you have already bought the 6 AWG wire, you may want to consider returning it or selling it and buying aluminum to save a bunch. But if you do that, you need to make sure that all connections are done appropriately. Connections directly to breakers and panels are usually not an issue, but connections between wires (e.g., between the already installed 6 AWG copper in your house and new aluminum for the outside portion) need to be done properly - you can't just throw the usual wire nuts on.