Was your general contractor going to do the work himself or was he going to outsource the work?
I have worked as a GC and I hire a lot of GCs. There are project costs and one-off costs. One-offs are things that are "fixed" that are sitting on their own island. Hanging a new door is a good example (unless you are retrimming the whole house).
This advice might be too late but maybe it helps you in the future or someone else. You should have been very up front with the GC for the one-off stuff. Get a quote, get a quote from the handyman, then figure out how much extra you would pay the GC.
Here is the deal with the GC. He is taking on the liability of material costs and extra labor to do job correctly. So in your example if door is installed wrong, he would ensure it is redone or not paid. If door is damaged during bad install that is on him. I am generalizing here - but if you hire a handyman you are pretty much getting whatever he does. Most will probably be fine... but what if yours isn't? Also why not hire someone from a handyman service so they have liability? Well that service is just a GC and will upcharge - or give you super low quality contractors.
So I think you made a mistake in not having an honest discussion with GC. I am blunt with the GCs and go through similar scenarios all the time. We have 6 extra small jobs. The GC quotes me $1700. I got a side guy that says I will do it for $1000. I tell the GC - "I got a quote for $1000, you are at $1700. I will give you $1250 or it is going to my other guy. I understand you have to make money and don't want to put your guys on jobs that make you less." - Notice I will probably offer my GC about 20% more than my lowest bid. He has more liability. I also have more liability if he doesn't accept - in that you don't want an unhappy GC.
Boom. Done. No animosity. Most of the time the GC will take the work for much lower just to keep his crew busy during issue times (forgot a plumbing part... well the two guys helping can install door while it is sorted). If the GC doesn't take the jobs, he had his chance. Now with handyman on site, you are the GC of the handyman and you basically have to make sure he is out of everyone's way. So for all intents and purposes you should not be leaving handyman alone at house working in the same house as GC for more than 20-30 minutes.
(I don't even leave two different GCs doing completely different work at a house alone for more than a few minutes unless I personally know both parties well.)