I have a small window in my bathroom that is high up near the ceiling. It's pretty useless, but since the bathroom is always really cold I think it is letting cold air in. I'd like to prove this theory by covering it up with some sort of temperature proof material. What do you recommend? I rent, so it can't be anything too damaging. I'd also prefer something that is not really ugly like plywood. Any suggestions?
I'd first check for leaks. Even a tiny crack can let a lot of cold air in over time. Caulking would be the proper solution, but since you're renting, foam weatherstripping tape may be a quicker/cheaper solution.
Next, if the window is single-paned or otherwise just poor quality, the cold could be coming straight through the glass. (This is probably more likely.) If you don't need the window at all and don't mind blocking its lighting/venting, you can get insulation sheathing for around $10 and cut it to size.
If you do need the light from the window, there are various window insulation kits available on the market for $5-10. Essentially it's shrink wrap taped around the window. Then use a blow dryer to even out any wrinkles.