I agree, you are undersizing the wires, and dramatically undersizing (cheating yourself out of) panel spaces.
Spaces, spaces, spaces!!!
I certainly don't need to tell you how frustrating it is to be out of panel spaces, since it just happened to you. We run into this every week - someone painted into a corner because the last guy just needed to chintz out. Gosh, I hope they really enjoyed the latté they bought with the savings on shorting you panel spaces!
Panel spaces are cheap! Regrets are expensive. Because of this, we have a saying: Go BIG or go home.
It's a trivial amount of money compared to total project cost, and it guarantees you never have that problem again. EVER.
So think more like a 30-space panel. Seriously. You'll never be back here going "I ran out of spaces again".
Also, double-stuff breakers are all but outlawed today, so you can't rely on "2 breakers per space" like you used to - that "circuits" number on most panels is bull-oney.
Remember to select a panel that provides accessory ground bars with the panel, otherwise you'll have to buy them. These are common "main lug" panels. You must separate neutral and ground in the subpanel.
Connect panels with conduit
Believe me, you'll thank me later for this...
Anytime you mount panels right next to each other, always connect them with several pieces of very short conduit, including a large one. Wherever you can find that the knockouts line up. Go big as you can. You can make these out of
- Rigid offset conduit nipples (if the panels set that close)
- plain Rigid conduit nipples with 4 conduit nuts
- short lengths of EMT conduit with connectors on each end
- PVC conduit... but then, you have to run a ground wire.
- Flex is OK, but not a valid ground path
- Preferably under 2' long (helps a lot)
Metal conduit provides the ground path, so you don't need a ground wire.
Now, this allows you to easily move circuits from one panel to the other. You simply use wire-nuts to extend all the wires except safety ground into the other panel. Including neutral. You can leave safety ground in the original panel, but you must extend neutral because AC power. The several conduit pipes make this very easy! Just go to the nearest one and pop the wires through.
You can use an NM cable with jacket if you really want to, but the most correct thing to use is THHN black and white wires.
As for the 3 main wires (hot-hot-neutral), those can be individual THHN wires and simply come through the big pipe. Bring all 3 through the same pipe. (Safety ground can use a different pipe if you need to wire it).
Wire sizes: 100A feeder
Now, you're off the rails quite a lot on wire sizes. Whatever website or personality you got that from, fire them :)
Here's how we actually determine your wire size needed:
I know that's a busy chart. Firstoff, can't use 90C at all, that's for industrial. At 100A or larger, everything gets to use the 75C column. So that's easy: our only remaining choice is copper or aluminum.
Forget the bad vibes you heard about aluminum, those only ever applied to small branch circuits (15-30 amps) - for feeder, aluminum has always been a good choice. Also, the panel lugs are aluminum because Al lugs play nice with both Cu and Al wire... so if you think copper avoids a dissimilar metal problem, wrong :)
"It's only 4 feet" - sure, I get that logic, but I want you spending extra money on more panel spaces, not copper. Novices who insist on copper wire and then get a tiny panel are a trope around here lol.
So #1 aluminum or #3 copper for your 100A feeder. Easy peasy. I would use three THHN individual wires, and white tape to mark the neutral. Nobody cares if the hots are black+red, but you can mark one of them red if you really want to.
Wire size: 50A Shed feeder
Since we're below 100A, some cables are only good for 60C. That includes NM/Romex cable (which you can't use outside, underground or in outdoor conduit) and UF-B cable (the gray stuff). So this makes the chart a little more complicated.
So you can see where this makes UF cable kinda suck because you must use a larger wire size. (#6 Cu or #4 Al). It also needs to be trenched 25" (for 24" of cover on top).
There are other direct-burial types that allow 75C running, but they generally start at #6 size. So #6 aluminum would probably be the most economical choice there.
PVC conduit is easier, only 19" depth (18" of cover), and then, cheaper wires. (#8 Cu or #6 Al).
Rigid conduit is the easiest to trench, only needing 6" of cover. You can trench it with a garden trowel and don't have to call 311 for a site survey... but it's VERY expensive. So trade off the cost of Rigid conduit vs cost of renting a trencher + risk of slicing through other utilities.
Rigid serves as the ground wire, so you can get your project done with 3 wires and smaller pipe.
Note that if you use individual wires, and neutral is #6 or smaller, you must use native white wire - you can't use black and mark it with white tape.
Due to a separate rule, for <=60A feeder, ground must be #10 copper or #8 aluminum. Again if it is #6 or smaller, it must be natively green or bare - no phase tape allowed. Bare aluminum is not allowed.