You can purchase "insulating concrete forms" made of foam that are designed and reinforced to withstand concrete pressure (if not damaged while being installed - there are some horror stories of "blowouts" while filling these in above-grade applications. Less of an issue in a trench, but don't damage them anyway.)
I agree with FreeMan's comment regarding the odds of your trench wall being flat enough that you don't just crack loose foamboard to bits.
Obligatory note to use reinforcing steel (it's cheap compared to the whole cost of the concrete job, and makes the concrete so much more durable/strong) and to provide a connection to the steel for an "Ufer" ground (concrete encased grounding electrode) near where your electrical panel will be located. It's one of the best grounding electrodes you can have, and costs practically nothing if you remember to install it when pouring concrete. The ties that hold the ICFs together have convenient notches to hold reinforcing steel in place.
As for doing this "without excavators" - you can certainly cancel your gym membership until the job is over. They can be rented (get some training and be very, very careful) or hired (with operator) and will make the job very much faster and easier - and safer. Can be done with Picks & Shovels but it is a LOT of work. You might need to ease into it gradually or you may get a lot of blisters that will slow you down (gloves advisable at first for that reason.) Realize that a 3 foot deep narrow trench can collapse, depending on your soils, and that may trap you by burying your legs. A bit deeper and it can kill you by preventing you from breathing. You'll definitely need to keep children out of the trench, as it's plenty deep to kill them in a collapse.