I have cedar siding and I've noticed some smaller holes. I've taken a chisel and verified that the hole is properly cleaned. There was some decayed wood, but I'm down to the hard stuff. Here's my question: how big does a hole have to get before I have to trade out the outdoor-rated wood putty for an actual piece of wood? Dime? Quarter? Silver Dollar?
It will depend on the type of product being used to fill the hole - some are better suited to small holes while others can handle larger holes. Products that don't (or resist) shrinking will be better for larger holes where the shrinking would be more noticable and might result in cracks.
Check out the products at your local home center and read the labels to find a product best suited to your needs.
As for taking size of the hole into consideration, both the diameter and depth are important to consider. A long shallow scratch will be easier to fill than a smaller but deep hole.