Node-RED Flow - Hubitat Performance Monitor


#1

Lately I've been working to track down what exactly has been causing my hub to slow down at times. As part of the troubleshooting, I wanted to maintain some graphs and logs of hub response times, and also to be notified when the hub begins acting up.

Great! What does it do?

  • The flow makes an HTTP request to the hub's App List page every five minutes and calculates the response time.
  • The response time is written to MySQL, InfluxDB, and a JSON log file on disk.
  • If the response time is between the warning and critical thresholds, a Pushover alert is sent
    • Notifications are sent only once per hour during waking hours and only once during the night.
  • If the response time is over the critical threshold, for 2 consecutive polling intervals (3 during the nightly maintenance window), the hub is rebooted.
    • To prevent continuous loops, a check is done to ensure that the hub is rebooted no more than once per hour.
  • Additional manual injection nodes are provided:
    • Manual hub reboot
    • A set of test injection nodes, one for each tier of response times
    • A simulated reboot of the hub for logging/testing purposes

Configuration

There are a few things you'll need to configure for everything to work as-is:

  • In the top right Configuration node:
    • criticalsForReboot: How many consecutive critical results are required for a reboot
    • criticalsForRebootDuringMaintenance: How many consecutive critical results are required for a reboot during the 2-3am maintenance window
    • hubIP: The IP address of your HE hub
    • thresholdWarning: The number of seconds above which the hub is in a warning state
    • thresholdCritical: The number of seconds above which the hub is in a critical state
  • In the Check Time subflow:
    • In the Am I Awake? node:
      • Latitude & Longitude: Replace with your own
      • Start Time: Time of day you typically wake up
      • End Time: Time of day you typically go to sleep
  • In the Log File subflow:
    • In the Test Log node:
      • File name: Set the full pathname for the testing log file
    • In the Prod Log node:
      • File name: Set the full pathname for your production log file

Requirements

The following additional nodes are used in this flow. They can all be installed via the hamburger menu -> Manage palette -> Install tab

  • node-red-contrib-advance-logger
  • node-red-contrib-config
  • node-red-contrib-influxdb
  • node-red-contrib-noop
  • node-red-contrib-pushover
  • node-red-contrib-time-range-switch

The Link

Special Thanks

Much inspiration and assistance came out of the thread linked below. Special thanks to @corerootedxb and @dan.t.


Hubitat > NodeRed > MySQL > Grafana (LONG READ)
How to create automatic backups of hub database
#2

NICE work Brian! Would you be willing to export/sanitize your flow? I'd love to piggy back on this.


#3

Nice Indeed!
I second @corerootedxb's request. I would love to import and refine this for my hubs.


#4

Your tact in pointing out that I'd forgotten the github link is appreciated. It's in the OP now. =)


#5

Tact?! TACT?! Have you met me (virtually)?! I haz no TACT! LOL :wink:

Thanks mate! I can't wait to get this imported! At the rate I'm going, I'm going to end up having to run multiple copies of NodeRed.


#6

I'm also working on a flow to toggle a virtual switch via the Maker API and listen for the event to come back over the websocket. I'm wondering if there will be a corellation with the times that this flow shows and which one turns out to be a better predictor of hub issues. Stay tuned!


#7

Its odd there doesnt seem to be any kind of slowdown at the 2am database rebuild on your chart. that's a known event where the hub slows down, I would expect that be evident on the chart, but yours doesnt seem to show it.


#8

The flow only queries once every 5 minutes. It's entirely possible that the database work was done between polls.


#9

@btk, I am seeing this in the debug console. Any ideas?

4/3/2019, 11:19:05 AMnode: Check Page
msg : error
"Error: getaddrinfo ENOTFOUND undefined undefined:80"


#10

Is your hub IP defined in the configuration node?


#11

Yes I think I did it right..


#12

Looks solid. Try clicking on the little green square that's attached to the left hand side of the config node. That should force the config data to be updated.


#13

I did click on that green square. Trying it again to see.


#14

Still getting the error.

I am hard coding the link for now.


#15

Interesting. Did you have to drill down into a subflow to get to the "Build URL" node, or did the export somehow flatten it all into the main flow?

Might be interesting to replace flow.get("$parent.hubIP") with flow.get("hubIP") and see what happens.


#16

I did have to drill into the subflow. flow.get("hubIP") still show the error.


#17

Thanks for this. I'm new to node-red, and linux in general.

Any chance you could show me what format to create the table for the database? I've got it all set up, but for that. Appreciate your work here. It's awesome!


#18

The MySQL database is the one that's created in the post linked at the bottom of the OP:


#19

Thanks, yes I had already added the table responsetimes to the DB hubitat_logging, I was meaning the format of the table. At present, I'm getting this:

image


#20

Here's the table definition:

CREATE TABLE `responsetimes` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `deviceId` int(11) DEFAULT NULL,
  `displayName` varchar(500) DEFAULT NULL,
  `value` decimal(10,2) DEFAULT NULL,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  KEY `responsetimes_deviceId_index` (`deviceId`),
  KEY `responsetimes_created_index` (`created`)
) ENGINE=InnoDB AUTO_INCREMENT=6039 DEFAULT CHARSET=utf8mb4;