Zigbee pairing issue with Nordic Semiconductor (ZBOSS stack) devices

I think this is the only way to get in contact with Hubitat developers. I’m debugging an issue some of my users are having when pairing SOMA branded shade motors to your Hubitat hubs. I have narrowed down the main issue to the way devices pair to Hubitat. Specifically the TCLK key exchange. I searched the forum for similar posts and they seem to confirm what I’m seeing. When pairing Hubitat your hub seems to always send the same TCLK key in Transport Key packet - namely ZigbeeAlliance09. This is rejected by ZBOSS correctly as per Base Device Behavior (BDB) specificiation (v3.0.1) section 10.2.5, step 9 in the Zigbee standard. This information was provided by Nordic support in this post: https://devzone.nordicsemi.com/f/nordic-q-a/128320/zigbee-samples-fail-to-pair-with-hubitat

Now on the C-8 hub I can get around this issue by using the "Pair while trying to avoid Zigbee 3.0 repeaters " option. It’s not ideal but at least there is a way my customers can work around the issue and still use Hubitat. But I do not see that option for the C-7 hub. I have both the C-7 and C-8 for testing so if there is anything you can suggest for me to try let me know. I would prefer not to break standards compliance for just this one older hub version but if there is no other way I may have to. It seems to be possible for me to add a special option for users to turn down security if they are using the C-7 hub to accept this default key.

Is there a specific reason you are using that key and why does the "Pair while trying to avoid Zigbee 3.0 repeaters " option get around this issue? Do you have any suggestions besides disabling that key check to make my device work with your hub?

Tiit

Tagging @mike.maxwell, Hubitat’s Zigbee SME.

looking into this.

In the HA 1.2 / legacy Zigbee PRO era, unique Trust Center link-key exchange was supported by the stack but generally optional and not widely enforced, largely for interoperability with devices that expected the default global key flow. Zigbee 3.0/BDB tightened that behavior into a commissioning requirement: the Trust Center-provided link key must not be identical to the device’s existing preconfigured key, and strict joiners may leave if it is.

On C5-C7, the platform already configures the legacy Ember NCP to generate a new Trust Center link key, but the NCP firmware appears to respond with the global default key anyway. That makes this a legacy radio firmware limitation/bug relative to Zigbee 3.0/BDB behavior, not something we can fix by simply changing the current join policy on the hub.

On C8 and later hubs, the default join process follows the the behavior of Pre C8 hubs, the currently, (and not really reflective of what it does) “Pair while trying to avoid Zigbee 3.0 repeaters” is the mode that returns a new TClink key.

So we can change the default to better support strict 3.0 compliance (return new TClink key), then make the option “Join legacy Zigbee devices“, but we have concerns about doing this obviously.

Open to other suggestions…

All Shelly Gen4 devices in Zigbee mode behave the same way: issue the Leave command as soon as my C7 gives them the ZigbeeAlliance09 TCLK.

But it sounds like this behavior can't be changed on the C5-C7, given what the NCP chip is doing in firmware.

@mike.maxwell - Given the C8 is the new chip, and newer devices should be more and more ZB3, is it possible to change the behavior to default to using the new/unique TClink key, and then if that fails, automagically fall back and trying with the ZigbeeAlliance09 TCLK key? And then just eliminate the option entirely?

Just thinking out loud, at this point.

We have thought about changing the order, however once a device basically gives you the finger and leaves it needs to be factory reset, there is nothing available at this phase of the join process to distinguish the Zigbee version in use

Ouch!

Thank you for explaining this. So basically my only option to support the older hubs is to ignore this key check and proceed with the pairing regardless? Will this cause any issues with certification? I have not checked into Zigbee certification but Matter had some pretty extensive testing that covered the security parts quite strictly.

I will try building a test firmware to skip this check but ideally I would not have this active for all users at all times as it seems like it could lead to security issues. Not likely to be a real threat but could be nice to show users that we take security seriously. How likely is the behaviour of C-8 to be changed in the near future? I think we still want to direct C-8 users to use the random key with the workaround and only do the fix for older versions.

If you want to have it work with the older hubs you’re going to have it behave more like a legacy 1.2 Zigbee device.

You might face this same issue with other Zigbee coordinators, though I don’t have any specific examples.

In regards to the current product line, we likely won’t change the order, but have discussed being able to capture this specific join fail sequence and present a ui element with further direction vs leaving the user hanging with no input from the hub other than abject failure.

We will also rename the alternate join option, however this is still in discussion and which ever way we go, these changes won’t be out until the next major release, 2.5.1

I suppose there could be a way on the device side of falling back to 1.2 mode if it fails to join in 3, but I have no idea what effect this would have on certification

Another option could be executing a specific join mode on the device itself by leveraging some specific button sequence on the device that wouldn’t effect certification as you would select 3.0 mode on a standard factory reset

This is another example of Aqara FP300 leaving the HE network 24 hours after the initial successful pairing.

  • FP300 (NWK:0x99E5) sends a Node Descriptor Request
  • C8-Pro responds
  • FP300 sends a Request Key command to the HE coordinator (Dst:0x0000)
    • Destination PAN is the C8-Pro PAN Id (0x1E52)
    • Destination is the Zigbee 3.0 mains-powered IKEA Tradfri outlet, which is used as a router between the FP300 and the hub (this router NWK is also confirmed by the Aqara devce in the diagnostic report)
      image

The C8-Pro hub does not respond ...

  • After 3 retransmissions without an answer, Aqara FP300 leaves the network...

Update/Clarification:
I mentioned the IKEA Tradfri outlet because it is the FP300 parent router / next hop in this capture.

The Request Key is logically addressed to the coordinator (NWK dst 0x0000), but on the air it is first sent to the parent router (MAC dst 0x3D8B). The IKEA repeater is therefore not the final destination and not necessarily part of the problem — it only shows the routed path FP300 → IKEA router → hub.

Also, when the FP300 is paired directly to the HE hub, I have not noticed such disconnections so far.

I’m a little confused about what’s happening here, are we thinking that the hub returned the ZigbeeAlliance09 tclink key and that’s why it left?, and maybe that just didn’t show up in the trace?

I don't have an idea why the HE hub didn't respond to the Request Key message in this case, it was online and responding to other Zigbee messages a second earlier... If I manage to reproduce the same problem again, I will share the info.

If possible keep your sniffer close to the hub during the capture, and if you capture the exchange pm me the wire shark exchange.