A week or so ago I upgraded a set of 5 C5s for new C8 Pros. For the most part, the upgrade was really smooth.
The big exception has been that every 12 hours or so MakerAPI stops posting device events to its configured URL. Commands to MakerAPI still work in that the device is changing state and that state is visible in the interface. It is not, however, propagating through MakerAPI.
I use Node-Red pretty extensively and this failure is very disruptive... Dashboards are all out of sync, and actions that are triggered by a device changing state aren't happening.
A reboot of the hub resolves the problem, and MakerAPI resumes posting the changed states of devices... For another 12 hours or so.
There is nothing in the logs that is unusual and the debug logs of MakerAPI look identical before and during an episode.
Most curious about this problem is that it seems to be unique to the C8 hardware. As I've struggled with this, I pulled one of the new C8 hubs and reinstalled its old C5 companion that I was was previously using and and the problem has disappeared on that hub. Now about every 12 hours, 4 of the hubs (the C8s) stop sending device state changes via MakerAPI while one of the hubs continues to operate flawlessly. Again, rebooting the 4 afflicted hubs restores all functionality.
I have migrated from a C3, to a C5, to a C7, to a C8, and finally to a C8-Pro. Maker API has worked reliably through all of these upgrades over the years.
Did you keep the same local IP addresses for each hub after it was converted?
Have you opened the MakerAPI app(s) on each hub and clicked "Done"? This can sometimes get things back in working order.
Have you tried rebooting the hubs with the "Rebuild Database" feature checked? (This is similar to performing an automated Soft Reset on the hub, which can clear out any gremlins hiding in the hub's database.)
When you mention every 12 hours - is that from the time you reboot the hub? Or does the problem occur at the same time(s) every day?
Just a few ideas. I use two instances of Maker API, one to send data to Node-RED and the other to send data to Homebridge using the "URL to POST device events to" feature. Both have been very reliable.
Yes. Each hub kept the same local IP address after it was upgraded.
As I've been trying to resolve the problem I've definitely pressed "Done" on MakerAPI several times both in the course of trying to resolve this as well as some additional development I've been working on. Yes, I've also tried rebuilding the database.
When I say every 12 hours, I mean that it seems to take about 12 hours for things to go wrong again after I reboot the hubs to fix them from malfunctioning.
I too have used MakerAPI extensively and found it to be really solid... until this last upgrade. So strange that I see this on 5 out of 5 hubs, and it works great for you. You're definitely using MakerAPI to post data to Node-Red and Home Assistant? You're not just using MakerAPI to send commands from NR and HA to Hubitat?
I am also using Node-RED (all my automation logic is in NR) and upgraded from a C-5 to C-8. I am still on FW 2.3.9.200 though. Have you tried downgrading the FW version?
I just started posting devices and hub mode to HA. I also noticed that things started getting stuck in the mornings. (At noon, HA thought I was still in "Early Morning Mode" when the hub was actually in away mode) To fix, I would go into Maker API, select done and then go into the HA integration and reconfigure to get the latest information into HA.
Since I just started using the integration, I was assuming that it was something on my end that I had not confured properly. I don't know about prior versions on HE since I was not using it until the latest update. However, it could be related to the same issue.
Great question. As I dig deeper, my three main uses for Maker API are...
For an instance of HomeBridge, which definitely uses the "URL to POST device events to" feature.
and for Home Assistant, which also uses the "URL to POST device events to" feature.
BUT...for Node-RED, I am using a webSocket connection on Hubitat to allow Node-RED to see every event without needing to use the "URL to POST device events to" feature.
I have verified that both Homebridge and Home Assistant are receiving updates from my HE hub without any issues.
I use the web socket connection for creating DB entries for all log entries and I found that periodically, the websocket connection would drop and there would be no notification. I created a flow that automatically restarts the Node-RED flow and that appeared to fix the issue.
I had duplicated events with the websocket connection. Which required me to setup debouncing gates. Even then, a few duplicated events would get through. So I went back to using http POST .....
However, for me, that won't really matter too much as I am using Node-RED primarily to collect data and then send it to InfluxDB. A few duplicate temperature or humidity readings are not going to be even noticed!
Really shouldn't matter in that use case. I was using Node-RED for automation, and without debouncing, multiple events certainly made for some interesting experiences!
@ogiewon Yes, I've used that websocket to connected Hubitat and Node-Red, and you're right, it isn't affected by this problem. Events continue to flow via the websocket during the failure of MakerAPI that I've described.
The creator of the Hubitat Hub for Node-Red (node-red-contrib-hubitat) recommends against using the websocket and I've had some problems with it. (lost and duplicated events like @rakeshg ).
@tray_e That does sound similar to how I noticed it in the early days before I had it nailed down to this very specific problem. I wouldn't be surprised if your issue is related.
Incidentally, it happened again last night. Almost exactly 12 hours since their last reboot, 4 of my 5 my hubs (the C8s) MakerAPI stopped posting events while the 5th even now continues to work perfectly.
Is that domain something you use internally on your LAN with your own DNS server? It seems to be unresolvable for me.
You may need to turn off this setting to include alternate DNS servers. I believe it is on by default, but may not have been available on the C5 so got turned on when you migrated maybe?
@jtp10181, Yes, that domain is resolvable only on my internal network. You make a great point about turning off the "well-known" DNS servers or even better, taking DNS out of the picture altogether and putting an IP address in APIMaker's POST URL. Thank you!
In the mean time, I'm still having problems. I've taken one of my C8 hubs and factory reset it. Then I added just one virtual switch and MakerAPI. I have Node-Red turning on or off the virtual device once every 10 minutes and seeing if MakerAPI picks up the changes. I"m curious if I can see the problem with that minimal install. If so I'll repeat with a C5 hub and see if the problem is there too.
But in the mean time I'm going to change my DNS settings and with any luck that will fix the problem for good.
Just saw this thread, but from the first post I was wondering if your LAN DHCP lease time is 12 hours. I posted a thread here a while back where my C8-pro would lose local DNS resolution every time the DHCP lease was renewed. Never had this issue with my C7.
Options are to use IP instead of DNS, or in my case, I'm pretty sure that switching the hub to static IP got it to keep the local DNS server settings upon lease renewal.
I was able to test by modifying the DHCP lease time on my router to 5 minutes, then using the network test tool on the hub to try to ping the local URL. Five minutes after a reboot, the hub would lose local name resolution.
Thanks for sharing this info.
