It's called feeder not service, unless it has a separate meter and electric bill.
Your trench needs to be 20" to give 18" of required cover over top of the conduit.
At only 100' you don't need to worry about a wire size bump for voltage drop.
If you are running cable for your feeder, your numbers are correct. If you are running cable in conduit, that is masochistic and conduit is not designed for that; as such the conduit needs to be bigger than you think. (not least to make the pulling possible, though it'll really be an awful task any way you slice it).
On the other hand, if the conduit runs the entire way, you are better off running THWN-2 or XHHW-2 individual wires through it, and then conduit works very well. You are able to run the feeder wires at 75 degrees C, allowing a higher ampacity:
- 4 AWG aluminum THWN/XHHW in conduit maxes out at 70A breaker.
- 6 AWG aluminum THWN/XHHW in conduit maxes out at 50A breaker.
As far as a GFCI protecting the feeder, that protects the feeder proper from leakage (all the more reason to use the correct rated wire)... but that is usually not considered valuable. (well, I like it for hot tubs, pools and docks).
Since GFCI trips on power tools do happen, having the GFCI at the shed makes more sense. The $80 breaker you have already sunk money into could be moved there, by using a 2-space "subpanel" to house the GFCI. It could not be installed in a backfeed position on a normal panel, because GFCI breakers cannot be back-fed.