In my quest to the perfectly perpendicular and/or perfectly placed hole, I have tried multiple solutions but the truth is, for truly, 100% accuracy you need:
A high-quality drill press with a chuck that is made to tight specifications. Then, you need to calibrate it with a special dial indicator (even drill presses drill inaccurate holes out of the box). Must calibrate the table left-to-right and in many cases the front-to-back needs some work as well. It is the only way to make a hole truly perpendicular to the face of the piece you are working on.
Then, no standard drill bit will work on metal (for wood this is not as critical) without a bit of wandering: they usually deflect a bit and "walk" slightly to one side before penetrating the material. This is more so in hard metal than on soft alloys but it happens almost all the time, even if you start with a smaller bit and enlarge the hole with subsequent passes with larger bits.
To solve this you need a "Center Drill".
This are very short and hard bits made specifically for this purpose. You use them to drill a tine hole, always smaller than the final hole you need but it will give your final-size bit a depression where it will start to drill WITHOUT deflecting.
Now, this level of accuracy is only needed is some situations but the point of this post was to pass some info I have gathered by researching the subject in depth and then my own trial-and-error not to mention a lot of cash spend on tools.
I can say after all this money and time spent I can make a hole in almost any material right where I want it, less than a thousand of an inch off, without breaking any drill bits.
To top it off, you cannot make a perfectly round hole with a drill bit, if you want to take it to the extreme, you need to finish with a drill bit slightly smaller than the true hole size you want and then use a reamer (again, mostly for metal drilling and more pronounced with the larger bits).
I understand this is taking it into the machinist territory but I am not a machinist and enjoy seeing how accurate I can be in my work/hobbies, sometimes well beyond what is needed, just to see how far I can go.