The term you're looking for is EVSE, or Electric Vehicle Service equipment. You are correct; it is a relay, computer-controlled GFCI and a computer that talks to the charger on the car, including telling the car the current it is allowed to draw. This is a "soft setting" and is configured in the EVSE at commissioning time, either through DIP switches or in a special commissioning screen.
The proper charger inside the car listens to the EVSE data and charges at the rate authorized. If the car ignores this, the EVSE senses it and cuts power.
Therefore you are not under any obligation to have an EVSE be a particular size. You can simply determine the surplus ampacity available, and set the EVSE per Code requirements given that ampacity.
Start with a Load Calculation
using the NEC approved procedure for doing those. That is a science-based formula that determines probable loads for a given dwelling.
You do two Load Calculations, actually. One for the house's entire service, and the other for the loads inside the subpanel. Once you have finished that, you know how many amps of "headroom" you have to give to an EVSE.
For instance if you calculate to 144A on a 200A service, you have 56A of spare service amps. If you calculate to 24A on a 65A subpanel, you have 41A spare subpanel amps. This would call for a 40A EV charging circuit off the subpanel.
Provision the breaker and wires on this basis.
So continuing the 40A example, you wire the circuit with a 40A breaker and 40A wires (that being #8 copper or #6 aluminum).
At these large sizes there is nothing wrong with aluminum. There was an issue with 1970s small branch circuit wiring, but that isn't applicable here.
Then, derate the EVSE 125% / 80%
Any EVSE circuit requires a 125% derate, NEC 625.14. So you take the circuit size and multiply by 80% (the inverse of 125%). For instance, if you have a 40A EV charging circuit, you take 80% of it or 32 amps. This will be the actual charging rate.
Note that 32A x 125% = 40A.
They're not "singling out" EVs. This 125% / 80% thing is a requirement on any continuous load. NEC also imposes this on many other kinds of appliances that are arguably not continuous, to silence such arguments.
Now, this is configured into the EVSE. The EVSE manual will have a procedure for setting the maximum charge rate allowed; follow the procedure (a NEC 110.3 requirement). For instance, Tesla EVSEs have you set the breaker size (40A) not the actual-charge-rate (32A) and the EVSE figures out the 80% thing on its own.
The gory details: so much more than a relay.
If you want to be picayune, the EVSE doesn't actually limit current. The EVSE sends a signal (a square wave on the Control Pilot pin) which tells the EV how much it is allowed to draw. (32A in our running example). The EV onboard charger detects the signal and chooses to limit the current draw to less than that.
So the EVSE just passes on the message to the car, but the EVSE is the only place the charge rate can be set.
That is on purpose, as UL will not approve any setup where the consumer could change the max charge rate in software. It has to be "crack open the EVSE and change DIP switches" or some equally elaborate procedure.
It also means any car can be plugged into any EVSE and the right thing just happens.
If you have a limited amp allocation you want to dynamically share between two EVSE's, they have tech for that called Share2 that also works slick.