Were it my problem to solve, I would simply add a brass kick plate on each side. Screw it to the door. You can buy these for not too much money at the home center in a large enough size that it will span the hole. Add some insulation in the hole first, at least if it gets cold outside during the winter.
As I said, that would be MY solution. My wife would disagree. She would insist on something that would simply disappear. Sigh. So my next choice would be to use a plate on each side that I could get paint to stick to. A quick search for "kick plate" on Amazon shows many different ones, not all of which are shiny brass. Some of those have a painted/paintable surface, so you could buy a pair of them, and then just repaint in a matching color. This will be almost invisible.
If that too is still unacceptable, then I might look into filling the hole in with a body filler, then maybe use repair techniques involving fiberglass mesh tape and epoxy resin to span that hole solidly. You would get a surface that can be sanded smooth, then repaint the entire door as if nothing was ever there. This will take some work though.