[RELEASE] File Manager Backup & Restore

thebearmay · January 23, 2023, 10:43pm

App allows for the creation of a compressed backup and the ability to restore individual files or the entire contents of the local File Manager. ~~Requires HE v2.3.4.134+~~

Available through HPM or at https://raw.githubusercontent.com/thebearmay/hubitat/main/apps/fmBckRestore.groovy

672southmain · January 23, 2023, 11:09pm

Outstanding. So, if I understand the process, after playing around with it,

Backup creates an .hgz file in the Hub’s local File Manager system, and
Restore restores files from within an .hgz backup file in the Hub’s local File Manager system.
The .hgz file can be manually downloaded to a local computer’s drive, either to preserve the backup or transfer to another Hub’s local File Manager system.

Correct?

thebearmay · January 23, 2023, 11:10pm

A+++ go to the head of the class

dennypage · January 23, 2023, 11:20pm

Very nice!

Pantheon · January 24, 2023, 12:58am

thebearmay · January 24, 2023, 12:59am

Are you on HE v2.3.4.134?

Pantheon · January 24, 2023, 1:00am

Bingo.....I'll update.

thebearmay · January 24, 2023, 5:18pm

So a question for everyone. I didn't add it in the first release, but on the to do list is adding two additional options:

Scheduling of a periodic (daily?, weekly?) automatic backup
To prevent #1 from getting too out of hand, a retention period parameter that would automatically remove any *.hgz files after X days

Would this be useful, or should I just leave it in its current on-demand mode?

Pantheon · January 24, 2023, 6:04pm

I prefer manual removal. I'm not comfortable with automatic deletions.

sburke781 · January 24, 2023, 6:10pm

I'd like the option to have a schedule and a retention policy, so not dictating you have to have it turned on, allowing for the manual control.

sburke781 · January 24, 2023, 6:16pm

It would also be nice to have some filters for the files to backup, like those starting with a certain prefix, or a certain file type (extension). An exclusion list would also be nice, like the ability to exclude image files I have on my hub.

672southmain · January 25, 2023, 1:43am

Perhaps a bit off-topic, but I was surprised quite a while back when someone (perhaps it was I) asked whether hub backup/restore backed up and restored the File Manager files, and learned that the answer was that it did not.

I realize that this probably won’t ever change. Your utility is a valuable addition when moving to a new hub. My file manager files (e.g., Smartly files, Hub Info html tiles, etc.) rarely change, but I am glad there is now some way to back them up. Thanks.

2ac16mo · January 25, 2023, 2:59am

You are the master

thebearmay · January 27, 2023, 3:24pm

v1.1.0 Adds logic for scheduling backups and backup retention
Edit: Had a few spare minues so v1.1.1 Adds a Create and Download button for backups

Edit 2: One additional note, if you make changes to the scheduling section hit the ~~Save~~ button before exiting.

iEnam · January 28, 2023, 10:50am

This is great, thank you!
All functions seem to working, but file does not download when "create and download backup" is pressed?

thebearmay · January 28, 2023, 11:13am

v1.1.2 Changes

Download not always working fix
Corrected retention purge logic
Added location events for backup creation and retention purge
Added a button to manually run the retention check

iEnam · January 30, 2023, 5:07pm

I can confirm that I was able to successfully use the download feature today, did not work 2 days ago on v1.1.2.
However, I attempted to restore a single txt file and managed to elevate the hub load.
Restore progress was taking way too long and basically hanged!
Please review the error below, thanks.

Summary

thebearmay · January 30, 2023, 7:08pm

The restore runs quite a bit longer than the create and if you're going to run with debug enabled I'd suggest commenting out line 356 as that will increase the run time by quite a bit -- it generates a debug line for every byte of the file being restored - in your case it added over 135,037 lines into the log. Using a 7MB hgz file to test with I wasn't able to duplicate your issue, what was the file size and how many entries were the file you were attempting to restore from?

iEnam · January 30, 2023, 7:40pm

Could you please give me some idea as to how long it takes for full or partial restore?
I've tried to restore the Hubinfo.txt file @135kB from a hgz file size of 375kB when the hub froze.
Will test again with debug disabled. Thanks
EDIT: just tested again with debug disabled and it seems to be fine now, weird!

thebearmay · January 30, 2023, 9:10pm

Writing out every byte to the log is a labor intensive process. Fairly pleased with the backup creation time, but still working on getting the restore time down. Restore is taking about 100-500x more time than the back up at the moment.