Correct. An Outbuilding needs a main shutoff switch even if it has one breaker. It does not need a main breaker, but it absolutely needs a shutoff switch.
It is unclear whether a "connected by roof" building is an outbuilding or not. Code plainly says it's not an outbuilding and does not need a main switch. Your local inspector is the final word on the subject.
Spend some time in the Eaton price book, and you find out that if you optimize for "cheap" or "compact", the best way to get a main shutoff switch is to get a panel with a big switch that by wild happenstance is a circuit breaker too. We don't care about that, we just need a shutoff switch. We only care that its "circuit breaker" trip value does not unnecessarily limit us. If your supply is 100A, then any main breaker 100A or larger will suffice.
This thing is not a breaker for us, and coordinating the breaker trip is hopeless - expecting this local breaker to trip first for our convenience violates Murphy's Law.
The one special characteristic this main "switch" might have is GFCI -- using an oversize hot tub panel is one way to provide necessary GFCI protection to every garage circuit at once, at the cheapest cost. The problem is, hot tub panels are woefully small, though this is helped by our ability to use double-stuff breakers if AFCI is not needed.
And "small panel" is death to a project like this. A person who brings 100A of service to their garage means to run some 240V loads. Those go through breaker spaces like congressmen through taxpayer dollars. Most loads in a garage need to be GFCI with some AFCI, so these will be full-space breakers.
The upshot is, don't even think of a panel less than 24-space...