Floor insulation is not going to make the room significantly warmer by itself, but if your basement is unconditioned, it can help eliminate drafts. Underfloor insulation can also help prevent heat loss by retaining heat in the winter and preventing cool air from falling into the floor below during the summer. This can also help by creating a vapor barrier, preventing any moisture from reaching the bottom of the wood floor above. Fiberglas insulation is also an excellent fire retardant.
I would start by using some roll insulation with insulation supports for the floor.
Regarding wall insulation, I suppose that depends on where you live. I live in southeast USA. I have done a lot of remodeling here and have seen a lot of mold issues where people have insulated behind plaster. I have also seen instances where that hasn't happened, primarily in the mountainous areas near my home. I know that plaster walls were designed to be able to dry themselves. I also know that plaster is nearly double the thickness of standard 1/2" sheetrock. Therefore, the "R" value of a plaster wall would be at least (probably more than) double of drywall.
Consider these issues before insulating the walls:
Seal cracks in the perimeter of your attic access hatch, around the inside of windows, at light switches/outlets and door trim. Repair any damage to the plaster walls..