I have an old field stone basement (1870's) that is crumbling but not just the mortar, the stone itself. We have tuck pointed the mortar areas with some success. Our problem is: the previous owner worked in a shipyard in the 50's and painted some of the walls with blue lead paint. The paint has disintergrated the actual stone into a sandy mess. Any solutions for this problem would be welcome.
The mortar is crumbling because of moisture seeping through the walls from the outside, not from the paint. That is a common problem with fieldstone foundations. See questions about how to remove crumbled mortar and what kind of mortar to use.
I would be very surprised if the fieldstone itself is actually crumbling. If so, you need a structural engineer that specialized in old foundations. Can you please post a picture? (If the system won't let you, upload it to imgur.com or somewhere else, and someone here can edit the picture into your question.)
Working with the crumbled mortar and peeling paint will probably require protective equipment because of the lead paint. See this related question. Bring in a specialist if you're unsure about how to protect yourself.