I've got a dry wall that had some old looks to be latex paint on it. I'm in the middle of repainting it and I see a small edge of paint stickin out. I try to pick it off and a sheet of paint rips up. I keep trying to pick the edges and more keeps peeling up. How should I go about painting over this? I can't get a smooth surface because there's always an edge of old paint left waiting to be peeled up. And I don't want to paint overs surface that can just be peeled up again so easily.
It's a drag, but it sounds like you already know what you have to do: peel off everything that peels off easily. No use painting over an unstable substrate; the new paint will just peel off too and you'll have to start over. Might as well do it right the first time.
It would appear, from the description that you have given, that the old paint layer never did adhere to the wall surface very well. When you are applying the new paint to the wall old paint softens and whatever hold that it did have is lost. Similar to the comment offered by @TFK I would suggest that your real choice seems to be that you may have to peel all the remaining old paint before starting over. During removal it may be possible to use a damp warm sponge to soften the old paint instead of having to use new paint for that job.
Several things will have to be investigated before attempting to refinish the wall. One is to understand why the old paint may have not adhered to the wall. The wall may have been very dirty, oily or covered in lots of dust. In most cases it would be necessary to thoroughly clean the surface, possibly sand it some and then apply a high quality primer paint. Make sure to use a primer compatible with the surface paint coat that you intend to apply.