Completely noiseless is a high goal - it can be done, but may increase the expense considerably. Quieter than a badly designed hot air system is much easier.
In-floor radiant heat, embedded in concrete or gypsum cement, offers essentially silent heat delivery (so long as the pumps are properly sized) - the boiler or hot water heater providing the heat may make a small amount of noise, which can be reduced or effectively eliminated by isolating it or removing it from the house entirely, and piping only the hot water in. But the amount of noise may not enough to actually bother you - does your hot water heater irritate? Is it gas?
Radiant tubing in non-concrete installations is prone to making some noise (a few creaks, ticking, or the like) due to movement from thermal expansion/contraction, as is hot water baseboard heat. Again, far less than a hot air system, especially a loud one. Indeed, the electric oil radiator you describe as "silent" generally makes some small creak/tick noises in my experience.
A hot air system that is properly designed can also be fairly quiet. So exactly how much noise is too much and how much you are willing to spend to eliminate it does play a large factor in how you proceed.