I ended up using a utility knife to cut off the old glue then I washed the part. I also cleaned the underside of the sink and sanded old glue off. It turned out pretty well:
I used clear silicone (applied with a caulking gun) as the adhesive:
Because the silicone doesn't adhere very strongly, I followed the advice to find a way to put pressure on it. I cut a wedge that can go between the overflow and the cabinet:
This overflow drain is in the front of my sink, although I have another similar sink where the overflow is in the back. In this case because the overflow is in the front, I figured out how to remove the front panel of the cabinet to make access easier. Here it is installed with the silicone and the wedge:
I used a screw from the outside of the cabinet to hold the wedge in place. The front of the cabinet ends up covering up the screw, so it isn't even visible once everything is back together.
I then added some more silicone around the outside, connected the tube, and put the front of the cabinet back.
So far it is working for minor overflow problems. I can splash water down there.
I tried filling up the sink up above the overflow and it isn't perfect in that case. The overflow drain doesn't take water as fast as the faucet can put it in the sink, so eventually the sink will overflow. I didn't glue the top of the overflow because it looks like it isn't meant to be glued there. Because the water level can get high, that means that water eventually comes out the top of the overflow into the inside of the cabinet.
I think I have repaired this the best that I can, but it appears to be a very poorly designed overflow system.