As long as the gate is usually shut, and the purpose of the wheel is to
- prevent gradual sagging of the gate over time
- ensure alignment of the latch
and if the gate behaves acceptably without the wheel while it is open, you could replace the wheel with a small alignment device at the closed position.
Screw a teflon wedge to the ground next to the corner of the house. The idea is for the gate to climb up the wedge, into aligned position and for the wedge to support half the weight of the gate. Screw a block of wood to the outermost slat, alongside the one the wheel is mounted to, vertically, extending from the floor to butt up against the bottom horizontal member. That's the piece that should contact the wedge and take the weight. Trim the bottom so it aligns correctly.
Edit: Here's a picture of what I mean. The red thing is a block of wood that is screwed (red arrows) to the existing structural pieces to the strain gets spread out and is not applied to the facing. The bottom of the block should extend down below the bottom of the facing and should be quite near the ground. The green thing is a greatly exaggerated wedge screwed to the ground that would contact the new block. It will be much more subtle than what I drew, only raising the gate by millimeters (Unless it starts to sag badly).
If the gate is misbehaving as it swings open, you should add more bracing. In the long run the wheel is going to need a lot of TLC to keep doing the job you've put it to and that gate shouldn't need it.