so I moved into a new apartment last month. The water wasn't very hot so we thought it was just because the weather has been getting cold. Turns out the water heater has only been on pilot, so the water wasn't getting hot. Does this mean that the water heater was leaking case because the water was only luke warm? We didn't smell any gas.
Many gas appliances have a "Pilot mode" where they will just burn a pilot light, and ignore the thermostat setting. Typically a 3-position control reading "Off-Pilot-On" and it sounds like yours is probably of that type and was in "pilot."
Turning this into a hunt for a gas leak is just plain illogical.
Almost certainly what the tech flipped from "Pilot" to "on" (and was internally laughing all the way to the bank about.)