The size of wire you use will depend on a few factors, including how it is being run.
If you plan to use conduit to run both sets of SE cables, you need to derate them, unless the conduit is able to enforce proper spacing (which it likely isn't). For two sets of 3-wire cables (four with ground), that is 4 conductors (neutrals aren't counted since they only carry imbalance current in a split-phase cable). In that case, your derating is to 80% of wire ampacity for its temperature rating.
In this case, 2GA copper wire is appropriate because it services 130A, which derates to 104A. If you have them running separately, then you can use up to 4GA. See this table.
Lastly, you were concerned with voltage drop. I'd say don't be. 4GA over 20 ft has a resistance of 5mΩ, which means at 100A, you would get .5V total drop. If you include the return trip (40ft total length), that's still only 1V... Check out this calculator.
All of this, however, assumes that each subpanel has a separate feeder breaker in the main breaker panel. If you plan to feed-through or do a T-splice from one cable to two panels, you will need to size for 200A since the cables can carry the full 200A load.
Also see this question.