I’d like to remove the drywall built-in pictured. That includes the ceiling, side wall and floor section. My question is how can I determine if it’s load bearing and if there is duct work in the upper section? Any other items I need to be concerned about?
You will not [likely] be able to confidently ascertain without opening up the drywall at least at the top. Doing so will give you more of a "lay of the land" in terms of ductwork, electrical, and framing.
Something to keep in mind is that as the house settles some weight may transfer to the studs framing the alcove, intended or not. The linked question mentions ways to determine if the studs are carrying load.