Backup Flow?

I searched but got a little confused with the node-red topics.

Does anyone have a flow that can pull the backups off the hub and save them to a cloud service? Preferably google drive?

I found a sequence online that sets up a database for upload to Dropbox but I can't remember where I found it online. This is mine after tweaked for my needs. I don't know how the function node labeled "parse html cheerio" works though.

I can export the sequence if interested.

please do

Let's try this:

[{"id":"8fe80bc3.8d8f18","type":"cheerio-function","z":"9035aa0c.21e758","name":"parse html cheerio","func":"const tableSelector = '#backup-table'\n\n//const cheerio = global.get('cheerio')\n//const $ = cheerio.load(msg.payload)\nconst options = {\n rowForHeadings: 0, // extract th cells from this row for column headings (zero-based)\n ignoreHeadingRow: true, // Don't tread the heading row as data\n ignoreRows: ,\n}\nconst rowEntries = \nconst columnHeadings = \n\n\n$(tableSelector).each(function(i, table) {\n node.log("Something happened");\n\n var trs = $(table).find('tr')\n \n // Set up the column heading names\n getColHeadings( $(trs[options.rowForHeadings]) )\n\n // Process rows for data\n $(table).find('tr').each(processRow)\n})\n\nif (rowEntries.length === 0)\n return null;\n\nmsg.payload = {\n fileName: rowEntries[rowEntries.length-1].Name\n}\n\nreturn msg\n\nfunction getColHeadings(headingRow) {\n const alreadySeen = {}\n \n $(headingRow).find('th').each(function(j, cell) {\n let tr = $(cell).text().trim()\n \n if ( alreadySeen[tr] ) {\n let suffix = ++alreadySeen[tr]\n tr = ${tr}_${suffix}\n } else {\n alreadySeen[tr] = 1\n }\n \n columnHeadings.push(tr)\n })\n}\n\nfunction processRow(i, row) {\n const rowJson = {}\n \n if ( options.ignoreHeadingRow && i === options.rowForHeadings ) return\n // TODO: Process options.ignoreRows\n \n $(row).find('td').each(function(j, cell) {\n rowJson[ columnHeadings[j] ] = $(cell).text().trim()\n })\n \n // Skip blank rows\n if (JSON.stringify(rowJson) !== '{}') rowEntries.push(rowJson)\n}\n","outputs":1,"noerr":0,"x":390,"y":680,"wires":[["1fb51151.02e45f"]]},{"id":"562a8ab2.b11ba4","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"http://192.168.68.124/hub/backup","tls":"","persist":false,"proxy":"","authType":"","x":170,"y":680,"wires":[["8fe80bc3.8d8f18"]]},{"id":"1fb51151.02e45f","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"bin","paytoqs":true,"url":"http://192.168.68.124/hub/backupDB?","tls":"","persist":false,"proxy":"","authType":"basic","x":430,"y":600,"wires":[["416921.3835c6e","27560d30.1f5e42"]]},{"id":"a72d2620.9c98f8","type":"cronplus","z":"9035aa0c.21e758","name":"Run Backup","outputField":"payload","timeZone":"","options":[{"topic":"AT115","payload":"","type":"date","expression":"0 15 4 * * *"}],"x":110,"y":600,"wires":[["562a8ab2.b11ba4"]]},{"id":"416921.3835c6e","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":610,"y":600,"wires":[["9e414f03.cc2fc"]]},{"id":"27560d30.1f5e42","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":670,"y":700,"wires":[]},{"id":"9e414f03.cc2fc","type":"file","z":"9035aa0c.21e758","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"binary","x":750,"y":600,"wires":[["352e4f2a.5d889","fb1b02ca.3bbc9"]]},{"id":"352e4f2a.5d889","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"strip","params":[{"type":"str","value":"/home/pi/shared"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":950,"y":680,"wires":[["46ed5a9b.0abf94"]]},{"id":"fb1b02ca.3bbc9","type":"join-message","z":"9035aa0c.21e758","name":"","text":"The Hubitat Database was successfully backed up.","title":"HUBITAT DB BACKUP","url":"","notificationicon":"","joinConfig":"a9299c99.7096","x":1090,"y":520,"wires":[]},{"id":"46ed5a9b.0abf94","type":"dropbox out","z":"9035aa0c.21e758","dropbox":"","filename":"","localFilename":"","name":"Upload","x":1150,"y":680,"wires":},{"id":"a9299c99.7096","type":"join-config","z":"","name":"Stephens Cell Phone","register":true}]

Can't import. :frowning:

I have had that happen previously when trying to import. I am not sure why it sometimes doesn't work. Let's try this with flow not in a block quote.

[{"id":"8fe80bc3.8d8f18","type":"cheerio-function","z":"9035aa0c.21e758","name":"parse html cheerio","func":"const tableSelector = '#backup-table'\n\n//const cheerio = global.get('cheerio')\n//const $ = cheerio.load(msg.payload)\nconst options = {\n rowForHeadings: 0, // extract th cells from this row for column headings (zero-based)\n ignoreHeadingRow: true, // Don't tread the heading row as data\n ignoreRows: [],\n}\nconst rowEntries = []\nconst columnHeadings = []\n\n\n$(tableSelector).each(function(i, table) {\n node.log("Something happened");\n\n var trs = $(table).find('tr')\n \n // Set up the column heading names\n getColHeadings( $(trs[options.rowForHeadings]) )\n\n // Process rows for data\n $(table).find('tr').each(processRow)\n})\n\nif (rowEntries.length === 0)\n return null;\n\nmsg.payload = {\n fileName: rowEntries[rowEntries.length-1].Name\n}\n\nreturn msg\n\nfunction getColHeadings(headingRow) {\n const alreadySeen = {}\n \n $(headingRow).find('th').each(function(j, cell) {\n let tr = $(cell).text().trim()\n \n if ( alreadySeen[tr] ) {\n let suffix = ++alreadySeen[tr]\n tr = ${tr}_${suffix}\n } else {\n alreadySeen[tr] = 1\n }\n \n columnHeadings.push(tr)\n })\n}\n\nfunction processRow(i, row) {\n const rowJson = {}\n \n if ( options.ignoreHeadingRow && i === options.rowForHeadings ) return\n // TODO: Process options.ignoreRows\n \n $(row).find('td').each(function(j, cell) {\n rowJson[ columnHeadings[j] ] = $(cell).text().trim()\n })\n \n // Skip blank rows\n if (JSON.stringify(rowJson) !== '{}') rowEntries.push(rowJson)\n}\n","outputs":1,"noerr":0,"x":390,"y":680,"wires":[["1fb51151.02e45f"]]},{"id":"562a8ab2.b11ba4","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"http://192.168.68.124/hub/backup","tls":"","persist":false,"proxy":"","authType":"","x":170,"y":680,"wires":[["8fe80bc3.8d8f18"]]},{"id":"1fb51151.02e45f","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"bin","paytoqs":true,"url":"http://192.168.68.124/hub/backupDB?","tls":"","persist":false,"proxy":"","authType":"basic","x":430,"y":600,"wires":[["416921.3835c6e","27560d30.1f5e42"]]},{"id":"a72d2620.9c98f8","type":"cronplus","z":"9035aa0c.21e758","name":"Run Backup","outputField":"payload","timeZone":"","options":[{"topic":"AT115","payload":"","type":"date","expression":"0 15 4 * * *"}],"x":110,"y":600,"wires":[["562a8ab2.b11ba4"]]},{"id":"416921.3835c6e","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":610,"y":600,"wires":[["9e414f03.cc2fc"]]},{"id":"27560d30.1f5e42","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":670,"y":700,"wires":[[]]},{"id":"9e414f03.cc2fc","type":"file","z":"9035aa0c.21e758","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"binary","x":750,"y":600,"wires":[["352e4f2a.5d889","fb1b02ca.3bbc9"]]},{"id":"352e4f2a.5d889","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"strip","params":[{"type":"str","value":"/home/pi/shared"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":950,"y":680,"wires":[["46ed5a9b.0abf94"]]},{"id":"fb1b02ca.3bbc9","type":"join-message","z":"9035aa0c.21e758","name":"","text":"The Hubitat Database was successfully backed up.","title":"HUBITAT DB BACKUP","url":"","notificationicon":"","joinConfig":"a9299c99.7096","x":1090,"y":520,"wires":[[]]},{"id":"46ed5a9b.0abf94","type":"dropbox out","z":"9035aa0c.21e758","dropbox":"","filename":"","localFilename":"","name":"Upload","x":1150,"y":680,"wires":[]},{"id":"a9299c99.7096","type":"join-config","z":"","name":"Stephens Cell Phone","register":true}]

Nope.. same error

me too I also tried to "pretty print it in Notepad++ and it threw this error - error while parsing... not much help...

Don't block quote it - blockquoting in Discourse strips stuff. Use the pre-formatted text option </> instead.

block quote (example)

[{"id":"74ddd96a.038a4","type":"bigtimer","z":"3943b86.98c64c8","outtopic":"","outpayload1":"","outpayload2":"","name":"On at 11:30 AM","comment":"","lat":"29.977091","lon":"-90.143000","starttime":"690","endtime":"690","starttime2":0,"endtime2":0,"startoff":0,"endoff":"5","startoff2":0,"endoff2":0,"offs":0,"outtext1":"","outtext2":"","timeout":1440,"sun":true,"mon":true,"tue":true,"wed":true,"thu":true,"fri":true,"sat":true,"jan":true,"feb":true,"mar":true,"apr":true,"may":true,"jun":true,"jul":true,"aug":true,"sep":true,"oct":true,"nov":true,"dec":true,"day1":0,"month1":0,"day2":0,"month2":0,"day3":0,"month3":0,"day4":0,"month4":0,"day5":0,"month5":0,"day6":0,"month6":0,"day7":"","month7":"","day8":"","month8":"","day9":"","month9":"","day10":"","month10":"","day11":"","month11":"","day12":"","month12":"","d1":0,"w1":0,"d2":0,"w2":0,"d3":0,"w3":0,"d4":0,"w4":0,"d5":0,"w5":0,"d6":0,"w6":0,"xday1":0,"xmonth1":0,"xday2":0,"xmonth2":0,"xday3":0,"xmonth3":0,"xday4":0,"xmonth4":0,"xday5":0,"xmonth5":0,"xday6":0,"xmonth6":0,"xd1":0,"xw1":0,"xd2":0,"xw2":0,"xd3":0,"xw3":0,"xd4":0,"xw4":0,"xd5":0,"xw5":0,"xd6":0,"xw6":0,"suspend":false,"random":false,"repeat":false,"atstart":false,"odd":false,"even":false,"x":160,"y":640,"wires":[,["40f6c6a.50913b8"],]},{"id":"40f6c6a.50913b8","type":"switch","z":"3943b86.98c64c8","name":"on only","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":340,"y":640,"wires":[["75012d94.fea7e4","b839400f.b325d8"]]},{"id":"75012d94.fea7e4","type":"hubitat command","z":"3943b86.98c64c8","name":"Hubitat-M set time","server":"a22e0ea9.276dc8","deviceId":"132","command":"force","commandArgs":"","x":530,"y":620,"wires":[]},{"id":"b839400f.b325d8","type":"hubitat command","z":"3943b86.98c64c8","name":"Hubitat-S set time","server":"662851c4.3ccad","deviceId":"2330","command":"force","commandArgs":"","x":530,"y":660,"wires":[]},{"id":"a22e0ea9.276dc8","type":"hubitat config","z":"","name":"HubitatM","usetls":false,"host":"192.168.1.46","port":"80","appId":"1522","nodeRedServer":"http://192.168.1.4:1880","webhookPath":"/hubitat/webhook","autoRefresh":true},{"id":"662851c4.3ccad","type":"hubitat config","z":"","name":"HubitatS","usetls":false,"host":"192.168.1.36","port":"80","appId":"4489","nodeRedServer":"http://192.168.1.4:1880","webhookPath":"/hubitat/webhook2","autoRefresh":true}]

pre-formatted text (example)
[{"id":"74ddd96a.038a4","type":"bigtimer","z":"3943b86.98c64c8","outtopic":"","outpayload1":"","outpayload2":"","name":"On at 11:30 AM","comment":"","lat":"29.977091","lon":"-90.143000","starttime":"690","endtime":"690","starttime2":0,"endtime2":0,"startoff":0,"endoff":"5","startoff2":0,"endoff2":0,"offs":0,"outtext1":"","outtext2":"","timeout":1440,"sun":true,"mon":true,"tue":true,"wed":true,"thu":true,"fri":true,"sat":true,"jan":true,"feb":true,"mar":true,"apr":true,"may":true,"jun":true,"jul":true,"aug":true,"sep":true,"oct":true,"nov":true,"dec":true,"day1":0,"month1":0,"day2":0,"month2":0,"day3":0,"month3":0,"day4":0,"month4":0,"day5":0,"month5":0,"day6":0,"month6":0,"day7":"","month7":"","day8":"","month8":"","day9":"","month9":"","day10":"","month10":"","day11":"","month11":"","day12":"","month12":"","d1":0,"w1":0,"d2":0,"w2":0,"d3":0,"w3":0,"d4":0,"w4":0,"d5":0,"w5":0,"d6":0,"w6":0,"xday1":0,"xmonth1":0,"xday2":0,"xmonth2":0,"xday3":0,"xmonth3":0,"xday4":0,"xmonth4":0,"xday5":0,"xmonth5":0,"xday6":0,"xmonth6":0,"xd1":0,"xw1":0,"xd2":0,"xw2":0,"xd3":0,"xw3":0,"xd4":0,"xw4":0,"xd5":0,"xw5":0,"xd6":0,"xw6":0,"suspend":false,"random":false,"repeat":false,"atstart":false,"odd":false,"even":false,"x":160,"y":640,"wires":[[],["40f6c6a.50913b8"],[]]},{"id":"40f6c6a.50913b8","type":"switch","z":"3943b86.98c64c8","name":"on only","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":340,"y":640,"wires":[["75012d94.fea7e4","b839400f.b325d8"]]},{"id":"75012d94.fea7e4","type":"hubitat command","z":"3943b86.98c64c8","name":"Hubitat-M set time","server":"a22e0ea9.276dc8","deviceId":"132","command":"force","commandArgs":"","x":530,"y":620,"wires":[[]]},{"id":"b839400f.b325d8","type":"hubitat command","z":"3943b86.98c64c8","name":"Hubitat-S set time","server":"662851c4.3ccad","deviceId":"2330","command":"force","commandArgs":"","x":530,"y":660,"wires":[[]]},{"id":"a22e0ea9.276dc8","type":"hubitat config","z":"","name":"HubitatM","usetls":false,"host":"192.168.1.46","port":"80","appId":"1522","nodeRedServer":"http://192.168.1.4:1880","webhookPath":"/hubitat/webhook","autoRefresh":true},{"id":"662851c4.3ccad","type":"hubitat config","z":"","name":"HubitatS","usetls":false,"host":"192.168.1.36","port":"80","appId":"4489","nodeRedServer":"http://192.168.1.4:1880","webhookPath":"/hubitat/webhook2","autoRefresh":true}]

Trying that (fingers crossed):

[{"id":"8fe80bc3.8d8f18","type":"cheerio-function","z":"9035aa0c.21e758","name":"parse html cheerio","func":"const tableSelector = '#backup-table'\n\n//const cheerio = global.get('cheerio')\n//const $ = cheerio.load(msg.payload)\nconst options = {\n    rowForHeadings: 0,  // extract th cells from this row for column headings (zero-based)\n    ignoreHeadingRow: true, // Don't tread the heading row as data\n    ignoreRows: [],\n}\nconst rowEntries = []\nconst columnHeadings = []\n\n\n$(tableSelector).each(function(i, table) {\n    node.log(\"Something happened\");\n\n    var trs = $(table).find('tr')\n    \n    // Set up the column heading names\n    getColHeadings( $(trs[options.rowForHeadings]) )\n\n    // Process rows for data\n    $(table).find('tr').each(processRow)\n})\n\nif (rowEntries.length === 0)\n    return null;\n\nmsg.payload = {\n    fileName: rowEntries[rowEntries.length-1].Name\n}\n\nreturn msg\n\nfunction getColHeadings(headingRow) {\n    const alreadySeen = {}\n    \n    $(headingRow).find('th').each(function(j, cell) {\n        let tr = $(cell).text().trim()\n        \n        if ( alreadySeen[tr] ) {\n            let suffix = ++alreadySeen[tr]\n            tr = `${tr}_${suffix}`\n        } else {\n            alreadySeen[tr] = 1\n        }\n        \n        columnHeadings.push(tr)\n    })\n}\n\nfunction processRow(i, row) {\n    const rowJson = {}\n    \n    if ( options.ignoreHeadingRow && i === options.rowForHeadings ) return\n    // TODO: Process options.ignoreRows\n    \n    $(row).find('td').each(function(j, cell) {\n        rowJson[ columnHeadings[j] ] = $(cell).text().trim()\n    })\n    \n    // Skip blank rows\n    if (JSON.stringify(rowJson) !== '{}') rowEntries.push(rowJson)\n}\n","outputs":1,"noerr":0,"x":390,"y":680,"wires":[["1fb51151.02e45f"]]},{"id":"562a8ab2.b11ba4","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"http://192.168.68.124/hub/backup","tls":"","persist":false,"proxy":"","authType":"","x":170,"y":680,"wires":[["8fe80bc3.8d8f18"]]},{"id":"1fb51151.02e45f","type":"http request","z":"9035aa0c.21e758","name":"","method":"GET","ret":"bin","paytoqs":true,"url":"http://192.168.68.124/hub/backupDB?","tls":"","persist":false,"proxy":"","authType":"basic","x":430,"y":600,"wires":[["416921.3835c6e","27560d30.1f5e42"]]},{"id":"a72d2620.9c98f8","type":"cronplus","z":"9035aa0c.21e758","name":"Run Backup","outputField":"payload","timeZone":"","options":[{"topic":"AT115","payload":"","type":"date","expression":"0 15 4 * * *"}],"x":110,"y":600,"wires":[["562a8ab2.b11ba4"]]},{"id":"416921.3835c6e","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":610,"y":600,"wires":[["9e414f03.cc2fc"]]},{"id":"27560d30.1f5e42","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"prepend","params":[{"type":"str","value":"/home/pi/Shared/Hubitat/BackupDBs/"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":670,"y":700,"wires":[[]]},{"id":"9e414f03.cc2fc","type":"file","z":"9035aa0c.21e758","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"binary","x":750,"y":600,"wires":[["352e4f2a.5d889","fb1b02ca.3bbc9"]]},{"id":"352e4f2a.5d889","type":"string","z":"9035aa0c.21e758","name":"","methods":[{"name":"strip","params":[{"type":"str","value":"attachment; filename="}]},{"name":"strip","params":[{"type":"str","value":"/home/pi/shared"}]}],"prop":"headers.content-disposition","propout":"filename","object":"msg","objectout":"msg","x":950,"y":680,"wires":[["46ed5a9b.0abf94"]]},{"id":"fb1b02ca.3bbc9","type":"join-message","z":"9035aa0c.21e758","name":"","text":"The Hubitat Database was successfully backed up.","title":"HUBITAT DB BACKUP","url":"","notificationicon":"","joinConfig":"a9299c99.7096","x":1090,"y":520,"wires":[[]]},{"id":"46ed5a9b.0abf94","type":"dropbox out","z":"9035aa0c.21e758","dropbox":"","filename":"","localFilename":"","name":"Upload","x":1150,"y":680,"wires":[]},{"id":"a9299c99.7096","type":"join-config","z":"","name":"Stephens Cell Phone","register":true}]
1 Like

victory! it works

1 Like

Yep, that worked, but there are so many odd Nodes, it's pretty useless.

Screen Shot 2020-06-10 at 10.19.51 AM

just add to your pallette

yes, I know HOW.. it's just more work than I want to do ONLY to explore a concept :smiley:

Who are you calling an "Odd Node"??? :grinning:

You need the palettes for the cheerio-function & string nodes. The rest are for nodes that I added to customize for my needs.

  • Cronplus sets the timer to run daily
  • Join-message and join-config is for notifications using Join that is part of Tasker on Android
  • Dropbox-out is obviously for Dropbox so if you aren't going to use that, you don't need that palette.
1 Like

of course you do sorry if I implied anything else :slight_smile:

I guess, in the back of my brain, I started that other Node-Red topic to have a base level of Imported Nodes that would allow US.. the Hubitat Community to share concepts via Imports... so that new Node-Red users aren't lost getting NR to be functional vs seeing/understanding Flows.

I can share my flow that backs up all of the important Node Red files nightly if anyone wants to see it but its got a couple of Odd Nodes too. :grinning: :grinning: :grinning:

1 Like

I personally don't have a problem with the Odd Nodes thing.. I know how to find them, import them, etc. But it's a peeve with Node-Red I guess, that THEY haven't created a far more useful (to noobs) installation. Yea, I know WE would have to add the Hubitat palette, but those 6 new nodes to be imported might discourage some.

Only if the end goal wasn’t visible or desired. When one is importing a flow/sequence, both those conditions are generally met.

I know when @april.brandt and I were sharing flows, we both added nodes necessary to get imported sequences to work.