This is a really common question, that applies to many things: heat, air conditioning, dehumidifiers, lighting and even PCs.
"Isn't it better to just leave the machine running continuously? The energy costs of restarting it must be enormous, surely they must be more than the costs of leaving it running."
Almost universally, the answer is nope. "Leave it running" is just an excuse/rationalization. What's really true is you want the facility to be already comfortable when you return, but you are bothered by the "guilt" of leaving equipment running unnecessarily. So you find a way to "believe" you are doing the right thing. It's a convenient untruth.
WHY? Consider: when you turn off the heat/cool/dehumidify, it has a "ramp" period where the house gets worse. Then it has an "equilibrium" period where the house has hit bottom, and doesn't get any worse. Then another "ramp" period when you start the machine back up, to bring it to your preferred level.
First, let's look at those ramp periods. Surely they cancel each other out, yes? Not quite. Actually, A/C and dehumidifiers work more efficiently when conditions are worse. That's readily obvious on a dehumidifer; it's much easier for the unit to get from 80% to 70% than it is to get from 50% to 40%. An A/C coil also has a much easier time cooling air from 100 to 90 than from 78 to 68. Heat transfers more easily when temperature differentials are higher.
Second, for that same reason, insulation works better when there's less temperature differential across it. The houses loses much less "cool" at 90 than it does at 70.
Third, that equilibrium period is absolutely free. The house doesn't get any worse, even though you aren't paying to run the machine.
So go ahead and shut it off if you don't need it. If you are annoyed having to suffer through the down-ramp period, then consider a Nest style thermostat or other "smart home" type automation that lets you turn it on via a timer, or auto-learning process, or remotely with your phone.