Moved here a few months ago and the heat pump is junk. It has a 100 amp breaker out of the main box. Thought about putting in a sub panel off of this breaker. Looked at the wire and it is a 3 wire (hot hot ground) and not a 4 wire (hot hot neutral ground). Not a pro sorry for the weird wording. Would I just connect the 2 hots to the sub and the ground to the bar and that's it or would this be a problem?
Is it really ground?
Look closer. Some cables, that wire can be used for either neutral or ground. If so, ask your AHJ whether they will authorize to re-classify it as neutral and retrofit an actual ground. The best plan is to ask your local AHJ if they are comfortable with that. You can't retrofit neutrals, but you can retrofit grounds.
If you don't get AHJ permission, you will get a very, very rude and expensive surprise when you sell the house.
All subpanels must be fed with entirely separate neutrals and grounds. Avoiding this is out of the question. But you don't need neutral for 240V-only loads. All your heat-pump loads are 240V, that's why neutral was never run in the first place!
Is it really 120V/240V?
If it's a subpanel for the heat pump, then it's very likely that every single load is 240V, with no 120V or 120/240V loads present.
In that case, install a 240V-only panel. Use a subpanel of your preference, get lots of spaces because every breaker will need 2 spaces, install a ground bar, remove the ground-neutral strap since it is a subpanel, and connect nothing to the neutral bar, it will simply float. Then you install your usual 240V breakers - a 20/30 for the heat pump proper, 30/40/50/70A for the emergency heat, etc.
Label the panel very clearly "240V only" so some future handyman or HVAC tech doesn't lose his mind or do something stupid.
You could make it a 120/240 subpanel later by running a new main cable.
Oh, but I need a little 120V too
First, try changing your loads to 240V. Motors can be had easily either way. It's easy to get modern lighting that happily runs on 120-277V, 240V being within that range. Old sodium/mercury/halide lighting often has transformer jumpers for that.
Second, OK, you really need 120V. How much? For loads up to 1200 watts, obtain a 1.5KVA transformer, or forget that, just bring a branch over from your main panel. For loads up to 4000 watts, obtain a 5 KVA transformer used on Craigslist, typically $100. Feed the transformer 240V primary from a 240V dedicated circuit (so you can turn it off). Feed the 120V secondary to a tiny 30-space* subpanel with both 120V rails jumpered together. Fit your 120V circuits.
If you really need more than 4000 watts of 120V at that location, knuckle down and replace that heavy cable with /3+ground cable. And get aluminum - the lugs are aluminum and this will reduce corrosion, also it's far cheaper.
* That's an inside joke. I am one of several here who recommend rather large panels, so you avoid the major hassle of running out of space. Feeding a 1.5KVA transformer a 2-space panel is plenty, from a 5KVA, a 4-space panel will suffice.