I got a USB wall socket fitted. When it was installed, I tested the USB port with a tester with the switches off, but the USB outlet was live. Is this normal?
In the US, no the USB ports will not have power when the outlet is not powered. Most outlets in the US don't have power switches on them, so if you switch them off there is no way for power to be provided.
In the UK, outlets often have power switches. In that case, your USB ports are usually powered while the outlet switches are off. This is because the outlet assembly itself is always powered, but the switches only control the outlet and not the USB power transformer.
You said cheers in your original post, so I'm assuming you're probably British and have the UK type of USB outlet. Yes, this is normal.
Other answers are assuming that you are in the US, where that wouldn't be normal.
Sometimes power supplies/transformers can hold voltage in capacitors to make it look like they are on for a short time after they are unplugged. Rather than checking with a tester, plug a phone (or anything that actually consumes power) into the charger and operate the switch to see if it is still on when the switch is off.
It's not impossible for the USB portion to have power all the time, but I've never seen a USB receptacle that would allow for it in the US.
edit: UK outlets can have integrated switches where this is much more plausible. See the other answer for the UK...