Yeah, your plan to splice in a junction box in the basement is fine. The junction box will need 32 cubic inches, so I'd consider a 4-11/16" square box as a minimum. JRaef's idea of a Polaris connector is the right concept, but you'll be better off with a "mini-Polaris" such as a MAC Block.
Upsizing to #6 wire for the last 4 feet buys you nothing. You can just stay with 8 AWG aluminum.
8-2 cable is also not legal for ranges (unless it is UFSEU cable with a ground in individual strands that spiral around the outside of the conductors. There have been a fair number of incidents of installers using 8-2+ground cable illegally. These cables have a single ground wire that is just a normal stranded wire like the conductors.