Everything you have described fits with a single 12 AWG 20A circuit feeding the shed. That is OK, provided a few things are in order, but is not ideal. The 50' distance is not a problem - as far as voltage drop it can be treated as if the shed was just a room in your house.
Underground wire must be one a few very specific cable types or be individual wires run inside conduit (metal or plastic tubes).
If it is a cable, see if you can find any writing on the outside of the cable, which will tell us what kind of cable it is. Sometimes people have the wrong type of cable and it is a code violation and can lead to real safety issues.
If it is individual wires then you should have a 3rd wire for ground, unless the wires are run through properly connected metal conduit. Unlike cable, individual wires should have labels printed on each wire, which will tell us if the wires are the right type or not.
If all wires involved are 12 AWG then this can be on a 15A or 20A circuit. If some wires are 12 AWG and some are 14 AWG then this can only be a 15A circuit.
Receptacles in a shed should be protected by GFCI. That can be at the breaker or at a receptacle inside the house or at the first receptacle in the shed.
The problem with having everything on the shed on one circuit is not code, it is practicality. If a shed is primarily for storage then a single circuit for lights + occasional tool or charger usage is OK. But if you want to turn it into a workshop then you typically want to have more:
- Lights - separate from tools so that a receptacle breaker trip doesn't turn out the lights
- At least 2 circuits for receptacles for tools. That allow you to run two large tools or a large tool and a dust collector at the same time.
- Additional circuits for heat, air conditioning, etc. depending on seasonal usage.
If you have conduit in place (last comments indicated PVC coming out of the ground, but that may or may not indicate PVC conduit for the full distance) then you can replace the existing wires with a set of 4 larger wires (hot/hot/neutral/ground) and install a subpanel. That opens up a world of possibilities and allows for GFCI protection in the shed (no need to go back to the house to reset).
If you do not have conduit in place, any upgrades will require replacing the existing cable with a larger cable. 12/3 cable will allow an MWBC for 2 x 20A, which will double your existing capacity. But if you go up to 10/3 or larger then you can put in a much more functional subpanel. Since proper direct-bury is not cheap and is limiting (to change again you have to trench again), conduit is worth considering.
Those are just he basics. I believe with a subpanel you will also need to have ground rods at the shed, which is not the case with a single circuit.
A little more based on the panel pictures:
You have a Rule of Six panel. That is supposed to mean "maximum of 6 breaker flips to turn everything off". However, you have 3 double-breakers (great) and 4 single breakers (not good). That is a total of 7. 7 > 6 = code violation. From a practical standpoint, that is kind of OK, because the top section (labeled SERVICE DISCONNECTS) has everything and in an emergency it would be "flip everything off in the top section". But technically wrong.
The design of the panel (that's where the diagram is critical) is that the middle pair of slots (one uncovered, one with a strange cover) are not used. Everything else in the bottom section is full. So if you do want to add a subpanel, the way to do it would be to move one of the single breakers from the top to the bottom, replacing the existing 20A feed to the shed. Then put in a double-breaker to feed to the shed. However, that would still leave you with 7 flips in a rule of 6. Two ways to fix that:
- Move the 15A to the bottom. Replace the 2 20A in the top with a double-20A.
- Replace one of the 20A breakers in the bottom with a double-stuff/tandem breaker, letting you move another one of the upper breakers to the bottom.