I have a bolt with a thread pitch of .8mm and a diameter of 3.3mm. I am trying to find the correct size of nut for it, however the ISO thread chart says that a .8mm thread is M5, but should have a diameter of 4.2mm leaving me confused.
Judging by the pitch and diameter you've measured, what you probably have is an American UNC 6-32 screw.
There are many conversion charts on the interwebs, but if you refer to this one in particular you can see the diameter for a #6 thread is 0.138" or 3.5mm.
The 2nd number in the UNC scheme is the number of threads per inch, so taking 25.4mm per inch and dividing by 32 gives you 0.79375mm - suspiciously close to your measured 0.8mm.
Big box stores (Home Depot, Lowe's, etc.) usually have a board hung up in their hardware aisle that you can use to identify threads/sizes of bolts and screws. You can just take your screw in and check it on the board. It looks like this:
Search for "thread measuring gauge" or "thread size checker" if you want to buy your own to have at home.
The great thing about standards is that there are so many of them to choose from.
Context is key,
If you are looking at modern electronic/IT equipment then the screws are most likely to be either metric (which you have ruled out) or UNC. From the other posts it seems that your screw is a good match to the 6/32 UNC thread. 6/32 UNC is a very common size used for mounting stuff inside computers (infuriatingly so is M3 which is almost the same size but with a much finer thread pitch, so your often have to test-fit a threaded hole to figure out which size it is supposed to be).
On the other hand if you are dealing with vintage gear then there are other possibilities, for example whitworth also has a 6/32 thread which is similar but not quite the same as the UNC one (similar as in with a bit of brute force you can get them to fit).
The thread has several measures:
- Inner diameter
- Outer diameter
- Thread angle (if it is triangular)
- direction (left / right)
All of them are arbitrarily chosen and described in standards. Therefore we have pipe thread, withworth thread, UN thread, UNC thread, ISO thread, fine thread etc. The only bolt-nut compatible pair is the pair of very same standard and very same name.
Some standards have geometries close together. For example UNC and metric have same thread angle but they are derived from imperial units (diameter, pitch) and metric units, respectively. There are nut-bolt pairs that fits up tu three turns. Screwing deeper, it locks.
If it is possible, throw the bolt away and get yourself new set with diameter that fits.