First, forget copper wire at this distance and size, unless you own a copper mine. Aluminum is a better conductor (by weight and cost), and is a better match to the zinc plated aluminum lugs you'll be attaching to. Use the dielectric paste. Yes, there was an issue with aluminum in small branch circuits (1350 alloy on receptacles and switches rated for copper only), but this is feeder. The power company still uses 1350 alloy on service laterals, and if you let them install this feeder they will use it. But you're supposed to use AA-8000 alloy.
Of course you know to use individual THWN-2 or XHHW wires instead of a balky cable.
Putting in 3" conduit is brilliant and will increase your ease of DIY. Just keep in mind you won't be able to put data cables in there. Only certain (non-conductive) fiber optic cables are allowed. You can put other power wires in there, say for a switched load like a light or well pump, but if you do, you'll need to derate all the conductors including the big set. Keep that in mind, this whole conversation is about derating.
Your minimum conductor size will be #1Al. If you install that, you are legal because any segment of wire won't get hotter than is safe. However you have many segments, and may have some voltage drop. You may want to upsize wire for the convenience of less voltage drop.
The first question about voltage drop is "do we care?" (Or to be more precise, how much do we care). People "chicken little" over a 3% number, but we just had a lost-neutral in our apartment complex where the balance tipped 20 volts (so 100/140V), and nobody noticed for days. The stock rule of 3% is concerned with this segment, with other unknown segments adding other unknown 3%'s cumulating... but in this case, this is the big segment - there isn't anywhere on either end to lose much more. So the 3% rule aims for no more than a 6-8% drop globally, and that's a fair target, given that the in-main and in-barn wiring won't add up to more than a 1% drop. Keep drop under 7%.
Lastly we must avoid the great bungle of most people doing voltage drop calcs. Do the calc on your actual amp load, not the number on the breaker. The breaker is there to protect the wire. If you voltage-drop-calc derate to 150A wire, you'd be perfectly allowed to up-breaker to 150A. It won't change your actual load one bit. Think it through.
Speaking of that, do not fit a 100A panel. The main breaker in the barn panel is redundant, it's just there to be a shutoff switch. A 200A breaker will work just as well as a shutoff switch, and give you a lot more breaker spaces too. Around here, we are fanatics at recommending lots of spaces, so you never, ever come back asking how to squeeze more circuits into a full panel! Spaces are cheap, regrets are expensive.
