TLDR: DO NOT put a 30/50 where you plan! It can't go there!
More spaces are ALWAYS better, and cheap. Think Really Big.
Don't overload your service. You need a load calc; see Ed Beal above.
Use conduit, not cable, to connect the panels. Reroute branch circuits.
Pay close attention to what Ed Beal is saying about load calculations. However, where I'll differ is that "number of spaces available" is a completely unrelated matter. They are separate issues, and you should think of them independently. Break the problem into 2 sections.
DO NOT put a 30/50 quadplex across from a 40/40 quadplex. You'll overload the bus stab. (This. An easy error to stumble into when dealing with overfull panels. Another reason overfull panels can't be tolerated!)
Section 1. Number of spaces
TLDR: You always want as many spaces as you can possibly get.
Electricity is there to serve us and do what we want. As you know, limited spaces are a serious problem, and so we here recommend that you have a very comfortable number of spaces. A lot of novices think the game is "have JUST enough spaces for your needs" on the logic that spaces are expensive. They're not. Spaces are CHEAP. Running out of spaces - that's expensive.
I gather you partially grasp this already; since you're adding a 12-space when I'm sure a 2-4 space would suit your immediate needs. I'm saying it's the right plan, but take it a lot farther. I like to finish an "out of spaces" upgrade with 48-ish total spaces. If you have a 30, add a 20. If you have a 16, add a 30. However in your case, since you already have nearly 40 circuits, I'd go for 60 total spaces. Get another 30.
The "circuit" number (12 spaces/24 circuits) is baloney. That's because most new circuits require AFCI, GFCI or both, and those are not made in the "double-stuff" breakers, and never will be. In the case of your 30/50 quad that you want to use, you're relying on a) the 30A being grandfathered and b) the 50A being a feeder.
Now as far as your 50A "feeder" to the subpanel, there's no reason to be demure there, either. Wire is cheap when you're only going 3 feet. Feel free to go as large as your main breaker.
You can have any size you want (unless you're dealing with a split-bus panel). So feel free to go 125 instead of 50; it can even exceed your main breaker. It's max that out. You can have a 125A feed to the subpanel if you want to. "More" might help, can't hurt, costs fairly little. Especially when your wiring is 2 feet.
The feed breaker has no bearing whatsoever on what loads are actually drawn. Your electric bill will be exactly the same whether your feed breaker is 25A or 125A.
Section 2. Load on the service
TLDR: Don't overload your service.
Now this is what Ed is talking about. After all, you're not installing a new subpanel for no reason; clearly you want to add something.
When you add circuits to a service, you must revisit the load calculation for the service. That applies to all the circuits on the service, regardless of which panel they are in. In other words, putting circuits in a subpanel is not a "free lunch".
On the other hand, there is no penalty whatsoever for putting circuits in one panel vs the other. There is no penalty for a huge subpanel feed breaker. There is no penalty for plenty of spaces. You might as well max them out; it's cheap, and being demure buys you nothing in terms of the load calculations.
You need to get the sharp pencil and run your load calculations based on the circuits you plan to add (which you have not discussed).
Use EMT conduit to connect adjacent panels, not cable
Of course, cable is what you know. I get that. But metal conduit is really good to get to know. It's literally perfect for panels right next to each other, and it's usually easier - well, it's 2 easy steps instead of one PITA step.
The only hard thing in conduit is the funny thread-nuts; but you'll have to use those even with cable, because you must have a cable clamp/strain relief where the cable exits and enters a panel.
Look at the 2 panels for where the large knockouts are. Position the panels so the large knockouts on main and sub "line up". Then you knock out the largest knockout hole they both have... get fittings that size and a short length of conduit (called a "nipple", which is awesome because you get to say "nipple"). Goes together like LEGO... when it's all just right, tighten all the fittings. And you just made your safety ground connnections.
If you want to come out the side of one into the top/bottom of another, look at prefabricated 90 degree sweeps in conduit.
Now if the smaller knockouts line up, I like to have a couple of additional 3/4" or 1" (whichever the pre-punched knockouts allow) conduits between adjacent panels. That gives you freedom to route circuits coming into panel A, over to panel B. The hot(s) and neutral must come over; the safety ground can stay in the panel it entered.
For the feeder, get 2 hots and 1 neutral of the biggest feeder wire that your feed breaker will accept. (a few feet of wire is too cheap to care about; again a silly place to scrimp). If the wires are #4 or larger, neutral can be a black wire. Metal conduit is the grounding path.
Edit: Given what I see in your photo and comments
I would measure your box size, then get an Eaton BR "main-lug" panel of the exact same "box size". That means all the knockouts on the sides will line up. That also means probably another Eaton 30-space 200A panel. That is fine. This could include an Eaton panel with a built-in surge suppressor, as a "neat" way to add a whole-house surge.
Then I'd measure the joist spacing in the bay to the left and right of this panel. If one of them is the standard 16" on center, I'd put this second panel there and embed it in the wall, preferably to the right as it'll fit out real nice.
Next I'd do exactly as I said: drill holes through the joist to connect some knockouts. Your box size is X5 (if 29" tall) or X6 (if 34" tall) and their right side doesn't have a lot of knockouts, so if the joist is non-load-bearing, I'd connect both top and bottom large ones, and one of the smaller ones also. Your feeder wires will only need a 1" knockout, so you might be able to get it done with all smaller knockouts as long as one of them is 1". You can't just have sharp box edges and wood, so "sleeve" the passage with a 3" long nipple of Rigid conduit, with conduit nuts clamping it down. That takes care of grounds. An oscillating tool is perfect to remove 2x4 material for the passage. (also for cleanly cutting the drywall).
With the conduits installed, I'd pull that 30A quadplex, route its wires through a knockout, and land it in the new panel. If you don't have enough wire slack you can extend with wire nuts. The reason I'm going after the 30A quadplex (instead of the easier 30A dryer) is stab limits. I want to put a 125A there, and that'll have a 20A across from it on each stab. The dryer has an 80A stab load across from it, so putting a 30/50 there will set that bus stab on fire.
Now you have two spaces free. A 125A breaker goes in there, and #2Cu THHN wire - 2 hots + neutral - all 3 wires through the same knockout to the main lugs on the subpanel. Normally I'd use Aluminum, but we're only going 2 feet, so keep the wires smaller (and they'll just fit inside a 1" knockout). If you want to wire a ground, that can go through a different knockout.
The multiple conduit passages give you the liberty to land new circuits in either panel, and move existing circuits over to the new panel if you want to.
If you had thoughts toward a generator, panel purchase time is the time to make sure there'll be an inexpensive generator interlock available for it. Chosen well, it's a $60 bolt-on. You would make this new subpanel the generator-fed one, and move any circuits you want on generator into this panel.
Lastly if you ever wanted 400A service, the way that is normally done is two 200A "main panels" side by side. Which is partly why I'm suggesting two 200A panels side by side :)
