Box fill requirements basically make that a hard no.
#6 wires need 5 cubic inches each. The device (receptacle) they connect to adds two more, so you need 30 cubic inches just for that.
#12 wires need 2.25 cubic inches each, and the device they connect to adds 2 more, but you get to skip the ground (or count only 1/4 of it under 2020 rules) so 9.56 cubic inches for that.
A 4x4x2-1/8 would just fit the 14-50R. The 20 amp outlet is easy in most boxes so long as it's separate from the 14-50R (and then you count the ground as it'a not a shared box - so you need 11.25 cubic inches, though you'll appreciate more room since GFCIs are big.)
Depending on how you have protected the panel/outlets from the weather in your home-built pedestal will affect whether you also need to deal with exterior boxes and in-use covers. Since you mention that it's an outdoor load center, it may be the case that you will need exterior boxes (which will vary somewhat from the cubic inch sizes I've called out above for interior stamped steel boxes - just pick ones that are at least as large or larger than required) of appropriate volume and in-use covers for proper weather protection.
If using a quad breaker, you might as well put in an additional 20A GFCI on the other 20A feed.
If the terminals on both breaker and receptacle are rated for 75C (and they usually are for a 14-50R) you could reduce the cubic inch requirement quite a bit by using #8 THWN or THWN-2 (3 cubic inches per count) since you are running wires in conduit, not cable, so you can use the 75C column in the table rather than the 60C column. Then you'd only need 18 cubic inches for that, rather than 30.