This varies greatly from jurisdiction to jurisdiction. It also depends on what kind of work and how much.
In many locations, minor electrical work, in-kind plumbing replacement, interior construction that does not change the overall footprint or the number or types of bedrooms, bathrooms, etc, (such as the building of a closet in an existing room) need no permit. However, some locales may require new installations to come up to a newer code level than those that they replace. You really need to check with your local building department to see what is allowed without permit and when permits are required.
Failure to get a permit for work that requires one could result in a stop work order if the building authority learns of the project. It also could result in the property not having a valid certificate of occupancy, which could make selling the property difficult. It also might compromise your relationship with your lender. Finally, it may put your insurance coverage at risk if a claim is based on the work done in violation of permit regulation.
Often if work is not done based on a permit, it can later be corrected by filing a new application, submitting plans and paying a penalty in addition to the regular fees. But there is a risk that the plans will not meet the standards when the permit is eventually filed, and may fail, requiring a redo.
Your best bet is to check and see what is required before starting on a renovation. Then you can make an informed decision.