Chances are the end of your horizontal stack into the box has a lot of tabs that are holding it together. I would leave this in tact. I would remove the duct tape (that shouldn't be used for ducts) and release the two pieces of horizontal duct that are being held together with said tape.
These are probably just smashed into each other and chances are you won't even have to have tools (maybe a screwdriver if they put a couple of small screws in) to remove this. Once you break these two pieces apart you can add insulation on your wall.
Note about the insulation - Please please focus on the lack of insulation I see in your joists above the cement going all the way around. This is way more important that insulating the walls of your basement.
Once you have insulated you may need some tin snips to cut back an inch or two on one of the horizontal ducts. You also may be able to smash the duct into each other sufficiently - this depends on how deep your insulation is. Also if you don't want to deal with cutting metal most big boxes will cut a sheet before they sell it to you. Not sure which one I would take them but possibly the one on the right. But anyway these are dramatically easier to cut when unrolled.