As the elected leader of a state, the Governor has a list of responsibilities that should be delegated, passing of the responsibility to an elected authority, prior to disappearing.
Disappearing for 5 days, what could occur? Where recent grass fires needed action from Sanford, and a state actively in the Hurricane season, how would anyone decide who to call and who is in command. With this type of confusion, lives could be lost and more damage to property.
Could care less about his affair but if any state Governor just took off without establishing a chain of command that should be grounds for impeachment.