I am trying to put up floating shelves and they came with the self drilling anchors (look like an auger). I can not get the anchor to drill all the way into the wall it gets stuck when it gets to the threads. It is not on a stud, I checked for that. When I tried to screw It in with a lot of force the plastic of the threads started to bend. Am I doing something wrong? Or might this be the wrong type of anchors for my wall?
Assuming you do have plasterboard/drywall, I've found a couple of tricks to make these plastic anchors go in more easily.
The first is to start with a metal one, then swap it to the plastic sort. That only works if the threads are the same pitch (spacing), but the ones I can get easily do match. This is probably the most helpful in your case
The second is to start the hole first, so that the tip doesn't have to work so hard and as soon as you start twisting the threads can bite. You can either use an awl, or lightly tap the fitting with a hammer (I use a metal fitting again). Either way, this breaks through the paper, any skim coat, and accumulated paint without chewing up the threads.
Note that stud finders can miss studs, and drywall can be fitted to thin battens which are harder to detect and often closer together. I have masonry walls with battens and plasterboard. If the fitting is going in by the thickness of plasterboard, then jamming, you're almost certainly hitting something. Using a metal fitting would give you certainty, at the expense of damaging the plasterboard if it does hit a stud or something else solid.
But fittings provided with products are often flimsy junk, so it may just be that they're too soft
If you pull out the anchor and then stick a screwdriver in the hole, does it stop between 1 and 2 inches deep? If so, you probably have drywall over brick or block. In which case there is only 1 to 2 inches of space behind the drywall and the anchor is too long. If what you're hanging is extremely light, you can cut the tip off the anchor.
The first thing that came to mind is that there is a nail-plate where you want to put an anchor. Any type of screw-in anchor should go into drywall easily. If they will not there is something behind the drywall.
Nail-plates are there to prevent nails or screws from being introduced into a component that is dangerous or essential, ( electric or plumbing ).
I would try to drill into the hole started by the anchor. Drill slowly and with a small 1/16 inch drill bit. If the drill hits something solid, push slightly for just a second or so. Retract the drill bit and look at the tip. If it has wood on it you hit a stud. You can just screw into the stud.
If the bit has metal or anything shiny on it STOP. The reason you can't get an anchor in the wall there is because there should not be one put there. Get a sticky anchor like Command Strips to hang what you need.
Another few possibilities for what's behind your wall:
Have you ever had plaster in the house, or do you have an older house? Plaster was commonly applied to boards called lath, which cover the entire wall from floor to ceiling. Sometimes drywall is installed over the lath during renovations, meaning that you could have drywall with wood behind it. Your anchor will not be able to go through the wooden lath, and it will feel like you've hit a wood stud but may also feel flexible or "spongy" since the wood is much thinner than a stud.
In a similar vein, I've seen drywall installed over all kinds of stuff, such as veneer paneling. This is thin stuff and would not necessarily be apparent if you've ever drilled or nailed through the drywall, but would absolutely stop a plastic anchor.
Other wood stuff lives behind the walls as well, such as cross-bracing between studs, and any number of renovation scenarios might lead to framing members being in weird places.
As others have suggested, drill slowly into your hole while inspecting the tip of the drill- if you see any metal or plastic shavings, stop and find another place to go into the wall (you've hit a drill plate, metal pipes, plastic pipes, etc.). If you only have wood, keep drilling. If the drill suddenly has no resistance and jumps forward into the wall, you've probably drilled through wooden lath or some similar thin impediment. Get a different type of anchor that expands behind the wall and you'll be fine. If you go the full depth of the drill bit and you keep biting into wood, then you're probably safe to just use a screw instead of an anchor.