This is a complex problem, and it's going to be difficult to provide an accurate answer without inspecting the equipment. So this is going to be more like a few comments, than an actual answer.
Water heater
I'll start with the water heater.
Old units aren't efficient
If the unit is older, it's likely not performing at peek efficiency, which is wasting energy. Updating the unit could save you money in the long run.
Tank heaters waste energy
No matter how much hot water you use, the heater is turning on from time to time to heat the water in the tank. The only way to eliminate this waste, is to install an on-demand unit.
Fuel source cost
Depending on the costs of different fuels in your area, switching fuel source could save money. Installing a gas fired on-demand water heater, could save you money long term.
Fireplace
If the fireplace doesn't have a fan, that is blowing heat into the home. You're quite possibly drawing more cold air into the home, than the amount of air the fireplace is heating. By using the fireplace, you could be blowing money out the chimney.
Also, the cost of wood is typically quite high, compared to other fuel sources.
Heat pump
Heat pumps use a lot of electricity, which in many parts of the country is an expensive fuel source. In temperate climates, heat pumps make sense because they're not typically used much. In climates where the temperatures often drop below freezing, heat pumps are virtually unheard of. This is because heat pumps aren't so good at heating homes, and often have to rely on supplementary electric heating elements.
Use the furnace on cold days
Since you have a gas furnace in the home, you can use that when the heat pump can't keep up. Depending on the local gas price, this could save you some money. In most dual fuel systems (like yours), if the heat pump can't keep up it shuts down and the furnace runs instead.
From your description, it sounds like the furnace and heat pump are running at the same time. This is not a good idea, and can actually cause damage to the system.
You'll want to configure the system (or have it configured) so that when the thermostat switches to auxiliary heat, the heat pump shuts down and the furnace kicks in.
Without actually being at your home, it's difficult to diagnose the problem. You might want to have an HVAC technician come out and evaluate the system, to help determine if you're wasting money.
What I did was contact my electricity supply and said "send me one of those energy monitor thingies free pronto"
Some months later it arrived.
I clipped the wireless current sensor around the main line/hot between meter and main-panel (consumer unit), inserted some batteries into the wireless transmitter and I was then able to see on the receiver/display unit how the power changed as I turned various things on and off.
![enter image description here](https://i.stack.imgur.com/BVjRh.png)
This particular unit display current usage and lets you look at averages for prior days/weeks/months but if you actually want to spend your own hard-earned money, you can buy energy monitors you can plug into a PC and do data logging, graphing etc.
Best Answer
Completely noiseless is a high goal - it can be done, but may increase the expense considerably. Quieter than a badly designed hot air system is much easier.
In-floor radiant heat, embedded in concrete or gypsum cement, offers essentially silent heat delivery (so long as the pumps are properly sized) - the boiler or hot water heater providing the heat may make a small amount of noise, which can be reduced or effectively eliminated by isolating it or removing it from the house entirely, and piping only the hot water in. But the amount of noise may not enough to actually bother you - does your hot water heater irritate? Is it gas?
Radiant tubing in non-concrete installations is prone to making some noise (a few creaks, ticking, or the like) due to movement from thermal expansion/contraction, as is hot water baseboard heat. Again, far less than a hot air system, especially a loud one. Indeed, the electric oil radiator you describe as "silent" generally makes some small creak/tick noises in my experience.
A hot air system that is properly designed can also be fairly quiet. So exactly how much noise is too much and how much you are willing to spend to eliminate it does play a large factor in how you proceed.