This is fine for 40A, as long as your EV charger does not require a neutral connection
A 6AWG aluminum wire is rated for 40A on 60°C terminations and 50A on 75°C (really 60°C/75°C) terminations. Your SE style U cable supplies two of these conductors, one for each hot, plus an uninsulated concentric sheath under the jacket that is in all likelihood the equipment grounding conductor (as your average hot tub has no use for a neutral), as permitted by NEC 338.10(B)(2):
(2) Use of Uninsulated Conductor. Type SE service-entrance
cable shall be permitted for use where the insulated conductors
are used for circuit wiring and the uninsulated conductor is
used only for equipment grounding purposes.
Given that SE cable is fine for indoor and outdoor usage as per NEC 338.10(B)(4):
(4) Installation Methods for Branch Circuits and Feeders.
(a) Interior Installations. In addition to the provisions of
this article, Type SE service-entrance cable used for interior
wiring shall comply with the installation requirements of Part II
of Article 334, excluding 334.80.
For Type SE cable with ungrounded conductor sizes 10 AWG
and smaller, where installed in thermal insulation, the ampacity
shall be in accordance with 60°C (140°F) conductor tempera‐
ture rating. The maximum conductor temperature rating shall
be permitted to be used for ampacity adjustment and correc‐
tion purposes, if the final derated ampacity does not exceed
that for a 60°C (140°F) rated conductor.
Informational Note No. 1: See 310.15(A)(3) for temperature
limitation of conductors.
Informational Note No. 2: For the installation of main power
feeder conductors in dwelling units refer to 310.15(B)(7).
(b) Exterior Installations. In addition to the provisions of
this article, service-entrance cable used for feeders or branch
circuits, where installed as exterior wiring, shall be installed in
accordance with Part I of Article 225. The cable shall be
supported in accordance with 334.30. Type USE cable installed
as underground feeder and branch circuit cable shall comply
with Part II of Article 340.
, your plan to reuse this for a 32A continuous/40A maximum EVSE seems sound provided the EVSE does not require a neutral (which this cable does not provide), and there is nothing in Code I can find that prohibits an outside branch circuit or feeder from re-entering the structure it originates in. Of course, you will need to use a 40A breaker for this (unless your EVSE has 75°C rated terminations, which will let you keep the existing 50A breaker and run the cable at 50A instead of 40A).