After caulking the four joints that make up a rim joint, can I put the rigid 2" XPS board close to the room (the beginning of the cavity), or does it have to be put right up against the joist. Because the house has had a past history of termite infestation (no termites seen for years after treatment) I don't want to permanently install the foam board so I can occasionally recheck for termites. I think, at least for me, once you have termites, you are always going to be concerned that they could come back.
In my house, a square piece of insulating board is up against the rim joist at the end of the bay and great stuff has been used around the edges of the insulating board. (My house was built in late 1800s)
edit: if you don't use foam, there's really no insulating value - insulation really only happens when air movement is halted.
2nd edit: foam is cheap. You can use it, then in the spring tear a few off to check for termites and then refoam. Use a sharpie to write the date on the foam the last time you checked it... just an idea.
I took a photo for you.