You could use exterior grade plywood, but you should still seal it on both sides with paint or clear coating suitible for exterior use. It still may be somewhat unsightly on the inside.
As an alternative, I have regularly used plexiglas type plastic to fill in the spaces along the sides of air conditioners. This avoids weatherproofing and appearance issues. It also give you a bit of a view to the side of the air conditioner (this was critical in my New York apartment where the peek-a-boo river view was in the only window that could take the air conditioner).
The easiest way to use this system is to remove the expandable plastic filler frame altogether. The accordian plastic is prone to leaking air anyway. While the filler sides do not provide much support, they do provide a bit, so it is essential to make sure the air conditioner is well attached behind the upper sash, prefereably with screws, and that the upper sash is blocked from opening with a bracket of some sort.
Cut the plexi to fit. You do need to seal the plexi on all four sides. One way to to this is to create a molding one side of which will press against the face of the plexi (on the inside surface) and the other side against the side of the window frame, the sill and the upper sash and against the air conditioner itself. Silicone sealant or similar latex glues can be used to glue the plexi and the molding. These can generally be removed fairly easily with a razor blade if needed.