I have some shelving in my closet and it's screwed into the wall with fasteners that look like this (see image). It's a smooth circle and doesn't have any type of hole of slot on top. Is this a screw? If so, what kind of screw is this and how do i get it out? I want to take this shelving down!
That's an expanding plug anchor as is commonly supplied by closet system manufacturers. Pull the nail out by the head (a locking plier works well), then pull the plastic plug out.
Here's a representative item:
ClosetMaid has a blog post on removal.
If it's actually a screw, use something like a Dremel with a cutoff wheel (or even a hacksaw if you can get one in there) to cut a slot, and unscrew using that. I've also successfully drilled a pair of holes and used a snake-eyes bit, when I didn't have room to get a Dremel at the head.
If it's not a screw you'll soon find out -- it will turn without unscrewing.
Another option is to file flats on it and use an adjustable spanner, but getting them both flat and parallel is tricky.
A third option is to drill a hole on axis and use a stud extractor. An easier variant on this is a left-hand drill bit, but they're rare. If you don't have a stud extractor a trick I learnt recently is to grind a bolt into a square taper and hammer that into the screw head, then unscrew the bolt.
This is most likely a plug like @isherwood suggests - nothing new there. I'm adding an answer to recommend a tool. If the OP doesn't happen to have a pair of vise grips on hand, a pair of screw pliers is in order:
These are a bit of luxury, but work like a champ, and are the perfect tool here. I picked mine up for about $20. No regrets- works great for stripped screws, too.