While admiring your planing, I agree with you that the outcome is questionable. The distance from Air-handler to the rooms in questions is simply too far.
First check if you can change the air handler speed setting (most of them have that option).
Second, consider putting in Zone-controller, that could redirect the air flow to your needs (less downstairs and more upstairs when needed.)
Third consider investing in Mini Split unit(s) that would specifically air condition the rooms in question.
Considering the complex work you were planing, the installation and cost of Mini Split might be attractive to you.
Last not least: Upgrading an existing panel to 200 amps (which is currently a required minimum standard) costs between $850-1,150 and your EV charger would be happy.