I live in upstate NY and my house was built in 1996 and it has the white thick foam insalation around the foundation. It looks awful and now I have bugs chewing it and building homes. Can I just take it off or do I need to replace it with something
I don't know for sure what "white thick foam" is, but any exposed foundation insulation should be covered with a finish material. It's generally the case that insulation below grade is fine, but that exposed to sun and weather degrades fairly rapidly.
You should not remove the insulation, as it was obviously installed for a purpose. What you should cover with depends on the situation, but some typical options are marine plywood, a parge coat of masonry, or vinyl sheets. Post more information about your home if you'd like a more specific suggestion.