There's an issue to be concerned about in high-crime areas -- they're plastic, so they can melt. Someone interested in breaking into your place can take a torch, heat up the frame of the window, and push the window or the whole sash in. Some vinyl windows aren't all vinyl, and have an aluminum or steel core, that might reduce this from being able to happen.
Also, vinyl isn't good for high-heat areas. A south or west facing window in a hot climate might heat up enough through the day that it'll warp. This is even worse in dry areas with large high/low shifts each day.
In colder climates, vinyl becomes brittle, which means if you're in a windy area with gets a lot of blowing wind, you could get debris thrown into the window which can crack the whole frame; and unlike other construction, you can't just replace the glass, you have to replace the whole sash.
Vinyl on its own isn't all that strong -- so there's a lot of vinyl involved -- if you're trying to get a lot of light into a room, you'll be able to get more glass into the area with just about any other material.
You also can't paint them easily. Whatever color they are, that's what they're going to be unless you replace 'em. This isn't typically a concern for most people, though, as most vinyl windows are white -- but they'll slowly yellow over the years, depending on the exact mix of vinyl. This is more pronounced in areas with higher UV indexes. (eg, high elevations with no shade trees)
Vinyl also outgasses. If you've got someone in the family who's sensitive to chemicals, this could be an issue.
That's not to say that there aren't different qualities of vinyl windows, but don't just buy on price -- look at internal metal stiffening, if it's welded or screwed together (you want welded), if it's fully encased in the vinyl (so it won't oxidize), and there's certifications now for vinyl windows ... but I think it's an industry effort, so I don't know for sure how reliable it is.