[RELEASE] HubConnect - Share Devices across Multiple Hubs (no longer SmartThings!)

storageanarchy · January 7, 2020, 7:39pm

Bug/Misfeature Report

I just learned thee hard way that Device Names and Device Labels cannot contain the forward-slash character (/) when connecting said device to a SmartThings remote client (via http). If it does contain this character, you'll get error asynchttpGet() request failed with error 400 errors every time an update is being sent. This probably also applies to ANY http-linked client, but I haven't verified it.

FWIW, I can attest that it doesn't matter for socket connections - the device I was having issues was on another Hubitat hub (Hubitat_HubConnect_socket_Client --> Hubitat_HubConnect_socket_Master --> SmartThings_HubConnect_http_Client). Device Name and Label were both "Guest Suite Temp/Humidity Sensor".

Either the name should be encoded before the transmission, or there should be a warning when selecting devices with the '/' in their name.

bertabcd1234 · January 7, 2020, 8:00pm

I've run into this as well (see post 1052). I wasn't sure if it was the server/client relationship that mattered, but what you discovered about HTTP vs. websockets certainly makes sense! My remote client was ST, so necessarily HTTP.

srwhite · January 7, 2020, 9:24pm

Trying to understand the nature of the issue..

Are you saying that SmartThings does not like device labels with slashes in them? It would be helpful to know exactly where does the error occur? The only time labels are transported between hubs is either saving devices or syncing a device, and in both cases they're conveyed in the response body as JSON strings where slashes should be legal. So I'm not sure where the issue really lies.

EDIT: For events, the server & remotes convert the event map into a JSON string then URLEncode it. So it's supposed to be encoded as '/' to %2F..

But the ST cloud could be decoding the URL and rendering it in the path. I know it is possible in Apache Tomcat for example...

I'll set up a test and see what's happening.

storageanarchy · January 7, 2020, 9:52pm

The error occurs on the HubConnect Server Instance for the SmartThings Client (I isolated the two errors below with blank lines...)

[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:25:18.682 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Bedside Keen Vent [temperature: 59.67 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:25:16.502 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Adam's Bedroom Keen Vent [temperature: 82.25 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:58.182 pm [trace](http://192.168.1.77/installedapp/configure/39)Received ping from ChezBurke SmartThings (ST0).

[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:45.831 pm [error](http://192.168.1.77/installedapp/configure/39)asynchttpGet() request failed with error 400

[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:45.178 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Guest Suite Temp/Humidity Sensor [temperature: 69.19 °F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:41.434 pm [info](http://192.168.1.77/installedapp/configure/39)Received device update request from client: [3211, type TempRH]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:40.046 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Guest Suite Keen Vent [temperature: 76.72 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:38.862 pm [info](http://192.168.1.77/installedapp/configure/39)Received command from client: ["Guest Suite Temp/Humidity Sensor": refresh]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:35.200 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Treadmill Keen Vent [temperature: 90.50 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:33.297 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoSensor: Bar: [motion, active ]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:33.187 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoTherm: Downstairs: [temperature, 68.9 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:32.962 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoSensor: Media Room: [temperature, 68.8 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:32.761 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoSensor: 1st Floor Foyer: [temperature, 69.2 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:19.458 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Bedside Keen Vent [temperature: 82.25 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:24:18.882 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Adam's Bedroom Keen Vent [temperature: 71.61 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:57.126 pm [trace](http://192.168.1.77/installedapp/configure/39)Received ping from ChezBurke SmartThings (ST0).
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:45.588 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Guest Suite Keen Vent [temperature: 72.07 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:44.605 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Treadmill Keen Vent [pressure: 21.92 inHg]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:43.564 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Treadmill Keen Vent [pascals: 74241.5 Pa]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:41.711 pm [info](http://192.168.1.77/installedapp/configure/39)Received command from client: ["Guest Suite Temp/Humidity Sensor": refresh]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:40.650 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Treadmill Keen Vent [temperature: 63.77 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:25.814 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Sam's Bedside Keen Vent [temperature: 80.82 F]

[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:20.591 pm [error](http://192.168.1.77/installedapp/configure/39)asynchttpGet() request failed with error 400

[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:23:19.828 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Guest Suite Temp/Humidity Sensor [temperature: 68.82 °F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:58.391 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Adam's Bedroom Keen Vent [pressure: 169.00 inHg]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:57.639 pm [debug](http://192.168.1.77/installedapp/configure/39)Sending event to ChezBurke SmartThings (ST0): Adam's Bedroom Keen Vent [pascals: 572315.1 Pa]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:53.920 pm [trace](http://192.168.1.77/installedapp/configure/39)Received ping from ChezBurke SmartThings (ST0).
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:32.451 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoSensor: Guest Suite: [temperature, 66.2 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:32.092 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoSensor: Sam's Room: [temperature, 64.4 F]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:31.796 pm [info](http://192.168.1.77/installedapp/configure/39)Received event from ChezBurke SmartThings (ST0)/EcoTherm: Upstairs: [equipmentStatus, idle ]
[app:39](http://192.168.1.77/logs#app39)2020-01-07 02:22:31.658 pm [info](http://192.168.1.77/installedapp/configure/39)Received command from client: ["2nd Floor Front": refresh]

srwhite · January 7, 2020, 9:54pm

I just saw that.. The event is reaching the ST device so it's processing the event correctly... Just throwing an error for some reason.

storageanarchy · January 7, 2020, 9:57pm

If the event gets there, it does not complete updating the data - I found this after several days trying to figure out why temperature updates on the physical device were making it to the Server, but not onwards to the ST client.

No errors appear in the HubConnect Remote Client logs on the ST side...

srwhite · January 7, 2020, 10:06pm

I am afraid right now it appears that my theory is correct.. The SmartThings cloud is performing a URLDecode on URL's before passing them to the mapping handlers.

Hubitat is properly encoding the URL... So the device name is "aaa/Test Button" and the '/' is being escaped to %2F as it is expected to be.

This blob is passed as a parameter for event data...

%7B%22name%22%3A%22pushed%22%2C%22value%22%3A%221%22%2C%22unit%22%3Anull%2C%22displayName%22%3A%22aaa%2FTest+Button%22%2C%22data%22%3Anull%7D

I'm going to have to figure out a clever workaround for this SmartThings platform issue...

EDIT: Actually I'll just drop the displayName field from the event data. The receiving end always uses the local device label so the data is redundant anyhow.

storageanarchy · January 7, 2020, 10:13pm

Will that work if both the device.label and device.name include the "/"?

srwhite · January 7, 2020, 10:16pm

I added those to the event data a long time ago because I wanted to save the hassle of looking up the child device and taking it's label. Events only convey displayName which is one or the other.

It's probably best to add the child device lookup and use that information instead of passing it a URL param since ST is decoding URL's before passing them to the mapping handler.

The only other way is to do some kind of BBCode-esque translation internally.

EDIT: Both options have pros & cons.. I'll figure out a best approach and work it into 1.7..

srwhite · January 8, 2020, 5:07am

HubConnect 1.7 Beta Update

I have received a good response to the invitation to participate in the beta so thank you for that. I haven't had a chance to respond to all PM's yet, so I apologize for that.

For the SmartThings issue reported today, I have a workaround for this quirk of the ST platform that will be included in the 1.7 beta.

Anyone who is concerned about monitoring hub slowdowns I've added an application-level ping monitor that could potentially help to monitor. The HubConnect server will send a message to the connected hub and the hub immediately echoes it back to the server. The difference is the RTT (in ms) for the message to make the full trip on the servers bidirectional websocket.

proxyJitterMS : 127
proxyLatencyMS : 12

The jitter value is a mesasurement (in ms) of the variance in the Hubitat websocket ping messages. This is a protocol-level event where the hub sends these at 120 second intervals. The server will track the interval between these. If this value is swinging wildly the hub could be experiencing heavy load.

This will hopefully prove to be useful in tracking hub slowdowns.

lairdknox · January 8, 2020, 6:03am

What am I missing?

I installed the HubConnect server components on my hub.
Installed the client components on a new hub.
Added a Zigbee RGBW bulb on remote and HubConnect driver on server.
Selected the bulb in the synchronize tab.

Server shows connected but I don’t know how to access the remote bulb. Both servers are running the current version (2.1.7.127).

lairdknox · January 8, 2020, 6:18am

Dunno what changed but now it is showing up.

storageanarchy · January 8, 2020, 12:09pm

FWIW, I know you try to avoid adding overhead whenever possible. Considering this, I would be happy with just a warning when I add the device. Or perhaps even an automatic translation of the name if the target is a SmartThings remote client. Some versions of Homebridge do exactly this, as Apple has a long list of "illegal characters" for HomeKit device names.

kilowatts · January 8, 2020, 2:49pm

Is anyone here using Grafana and Node-Red? I'm graphing attributes from 100% HubConnect virtual devices on the coordinator hub. It's been working very well so far until I reboot the coordinator hub - then Node-Red gets stuck. I'm wondering if there's a way to automatically restart the flows, or see how other folks are doing it. (maybe this is a question for the Node-Red thread)

srwhite · January 8, 2020, 3:24pm

I am not although it is on my bucket list, which means it's probably not going to happen anytime soon.

I know little about it, so pardon the questions... How does the integration connect to the hub? Is it looking at the hubs websocket or is there an app that catching subscriptions and sending the data?

I have a Nodejs logger for HubConnect that dumps data straight into a MySQL database which listens to the Hubitat websocket. I had issues with that not reconnecting after a hub reboot so I had to add some code to monitor the websocket ping intervals and take connection recovery actions if needed.

dan.t · January 8, 2020, 4:11pm

Yes I am

Yes there is.

You can send a HTTP POST request that has a special header set to restart the flows.
I have modified @ogiewon wonderful "HTTP Momentary Switch" to add the special header to it. My modified code is here:

github.com

danTapps/Hubitat-4/blob/master/Drivers/http-momentary-switch.src/http-momentary-switch.groovy

/**
 *  HTTP Momentary Switch
 *
 *  Copyright 2018 Daniel Ogorchock
 *
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 *  in compliance with the License. You may obtain a copy of the License at:
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
 *  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
 *  for the specific language governing permissions and limitations under the License.
 *
 *  Change History:
 *
 *    Date        Who             What
 *    ----        ---             ----
 *    2018-02-18  Dan Ogorchock   Original Creation
 *    2019-10-11  Stephan Hackett Added ContentType and Body inputs(very limited functionality)

This file has been truncated. show original

I created a new virtual device with that driver and set the following parameters:
"Device IP Address": IP address of your Node-Red Server
"Device Port": 1880
"URL Path": /flows
"POST, GET, or PUT": POST
"Content-Type": application/json
"Extra Header": Node-RED-Deployment-Type:reload

I then created a rule in Rule Machine that gets triggered when the Hub reboots and as action it turns on this new virtual switch.
That effectively restarts the flows in Node-Red and reconnects the event socket.
Has been working like a charm

Yes, Node-RED listens to the event socket, persists the data either in an InfluxDB or MySQL/MariaDB and Grafana reads the data from the database to visualize it.

kilowatts · January 9, 2020, 12:12am

Thank you.

ritchierich · January 9, 2020, 2:21am

Yes I am having similar problem and solved it:

troywehrle · January 9, 2020, 5:44pm

Hi all. My HA system on a single Hubitat, has been starting to experience slow downs and delays in executing scenes and responding to/representing events. So I purchased my second Hubitat. It appears to me that HubConnect is the better architecture for integrating them, so I went that approach and they are now integrated.

My biggest question and next step is how to partition the devices, rules, logic, apps, etc. between the two Hubitat's. I have read tons of threads here and studied the posts from @csteele and @srwhite to help me decide on the best partitioning approach. It seems you both differ in your approaches.

It appears one approach (#1) is to put all physical devices with resulting radio traffic/events on one Hubitat, then mirror them to the other Hubitat, which runs rules, apps, etc.

Another approach (#2) seems to split physical devices across Hubitat's, with rules, apps, etc. relative to the physical devices following them to each appropriate Hubitat. This seems less achievable, since rules, Dashboards, etc., often times need multiple devices to trigger and/or action.

The other approach (#3) I have seen is to partition them by radio protocol. This doesn't really apply to me at this time, because my system is all ZWave.

So I am left wondering what the best practices and experiences have been before I move forward. I am leaning towards approach #1 above, because it seems more achievable without total disruption of my system, but I am wondering if I will see the net result of an overall better responding system.

Your thoughts, please????

erktrek · January 9, 2020, 5:57pm

I have 3 "production" hubs - one for basement/first floor devices, one for the 2nd floor devices and a "controller" hub in my office. I use the first/second floor hubs to control devices and keep the rules local to each hub. For cloud devices like Alexa control and other misc stuff like Sonos etc I use the controller hub.

In terms of linking via HubConnect I try and keep it to a minimum of devices and apps because I prefer local hub processing. I do use Alexa app on my controller to control lights on my first & second floors via virtual switches.

I have been happily using HubConnect for a while and it's been great. I should add I have around 150 devices between floors of mixed type - Zigbee, Z-Wave (and Zw+)