GE In-Wall Z-Wave Outlet (12721) Duplicating Commands?

🛎️ Get Help
1

So, I've long had a suspicion there was something wrong with either this device or a repeater between these devices. I bought a cheap sniffer to scout out what was going on and it was so cheap it made me come to the wrong conclusion.

I next built a zniffer (now that a lot of the silabs.com tools seem to be available without a NDA) and it painted a different story. I got this capture:

image
image1344×381 67.1 KB

What you are looking at (I think, since I haven't been using the Zniffer software very long) is a BinaryGet from the hub to outlet (node 37). It has to jump through node 42 and 9 to get there. Once it gets there the outlet initiates the Ack back through all of the routing devices and then the hub says Ack back to the last routing device. That must be the protocol on the Acks. After the ack comes back through there is almost like a thank you on the last leg of the Ack back to the last hop.

Anyway, that looks all fine.

What doesn't look fine is that the outlet (again, node 37) turns around and initiates not one, but two, binary reports back to the hub even though there were no issues with the first one. So, the log reports that the switch is on twice.

image
image726×74 3.08 KB

This was a small but super clean example of what I'm seeing large scale sometimes with all of my outlets of this model and firmware.

Sometimes, instead of just a duplicate report they send multiple duplicates and even to the wrong devices and even switching the command completely. When this happens everything starts talking and sometimes I have to bring the hub down for the all of the devices to settle down.

When it happens it appears that the outlet doesn't keep talking during the entire collapse. It just instigates and then watches society collapse.

RSSI is 50 to 69. I don't think that is the RSSI that the devices see between each other but what my zniffer module sees from snooping the packets. Either way, it feels high enough because I'm able to see everything sent without corruption. First question, is there a way to ask a device what the RSSI is for its neighbors?

Second question, in the protocol is there any reason for the outlet to do this? It got a successful Ack from the previous binary report. I can't see any reason for it to send another one.

If it is part of the protocol I want to point out that I don't see it doing the second report every time. Also, that means the hub isn't handling it correctly and shouldn't have reported the second report. If it ISN'T part of the protocol, I don't think we can blame mesh in this case because we see the Ack messages sending and receiving correctly. Do we blame firmware? Are any other users with duplicate messages in the logs using these GE outlets?

1 Like
2

Do me a favor... switch to the basic z-wave tool driver.. and do a command class report ... and post the logs of it here..

3

I actually do this before starting on any driver.. It is an extremely handy tool..

4

I'll do you one more... I did that long before I had the Zniffer. I've already written a custom driver focused on the CCs that it supports. Here are the classes that were returned with the text descriptions for each.

def getCommandClassVersions() {
	[
    0x27: 1,  // Switch All
    0x25: 1,  // Switch Binary
    0x70: 1,  // Configuration
    0x72: 1,  // Manufacturer Specific
    0x73: 1,  // Powerlevel
    0x75: 1,  // Protection
    0x77: 1,  // Node Naming
    0x86: 1,  // Version
	]
}
1 Like
5

Wait.. is that every command class the device supports? or just the ones you implemented in your driver?

6

Every command the device supports. There was no documentation about which CCs this device supported on z-wave alliance so I had already asked the device ages ago.

You have to understand I had wrongly assumed that this was a hub or driver issue for the last few months. I've scoured the outlet parameter options to see if it was something I could change via a parameter. I'm pretty sure it's not. Switch All is an interesting thing if you look at what it does but even with it turned off... it still doesn't change anything.

7

i dont know

not to my knowledge, perhaps if the device isn't getting an ack back in what it considers to be timely, it's probably going to retry sending the report, however it is getting an ack in less than 70ms, which given two hops seems about correct.

1 Like
8

That was my best guess.. I was digging in docs to see if I could answer the first part.. But it doesn’t appear so.. Not to the extent of rssi for each neighbor node..

Although that would be handy

9

So I guess my next steps to troubleshooting this are to take one of the outlets out of the gang box and run it closer to the hub and watch it some more.

If it performs okay then it's a signal strength issue. If it doesn't then it's firmware. You would think the certification process would have caught a bug like this... but at the same time... I can see the packets without any issues.

1 Like
10

Yea. You can.. But as Mike pointed out it might be a timeout on the ack .. If the device doesn’t receive an ack in x ms it assumes the packet didn’t get there and retries.. This is speculation at this point.. But if you can I would really replace them.. Mains powered non-plus devices will slow down your whole mesh

11

It won't screw up without any routers in the mix (my guess here), and no they don't test in cert with routers...

2 Likes
12

The best thing I did was remove all mains powered z-wave pre-plus devices.. My z-wave network screams now..

1 Like
13

Isn't the Sniffer a factor too?? the fact that it can see the packets does not mean the outlet did. If line 13 was sent, as shown, and received by the sniffer, doesn't mean the packet made it to the outlet. That would cause a retransmit, I believe.

2 Likes
14

To me, the fact that I can see the Ack packets and no additional Ack packets afterwards tells me that the signal is okay. I just have a hard time believing something this mundane was missed in certification.

1 Like
15

If they didn't test with a repeater (two in fact), how would they catch this?
Also, so far I haven't seen anything specifying that get commands shall return one and only one response report...

16

Do you ever see other devices sending back two reports? And the weird thing is... seemingly waiting for the original report to be fully sent and acknowledged before sending the second report.

17

I havent seen this in my lab, nor in production.
The lab environment seldom has any routers in it.

The only dups I see are when a device sends two different reports for the same event and the driver processes them to the same Hubitat event.

18

Then do you think it is fair to call this a firmware issue on the outlet?

19

Sounds like it.

20

So, here's another case where the outlet looks a little more innocent because it was provoked but is STILL acting a desperate.

image
image1484×1030 196 KB

image
image1474×1101 224 KB

image
image1511×1097 215 KB

At first the hub tells 37 to turn off by way of node 42 and 9 on line 26954. The message gets to 42 and on line 26958 42 sends a message to 9. There shouldn't have been a route Ack yet but apparently something took too long because 42 sends another message to 9 on 26964. In the meantime, in row 26965 42 tells 1 that the first message to 1 didn't work with the route error. 1 says "Ack" 42 on line 26966.

So, even though the second message from 42 to 9 is still in the wind 1 decides to try a new route on line 26944 by way of node 18. This second route goes through very quickly on lines 26977, 26978, 26980 and 26981.

Now, since 37 is a Z-Wave product 1 wants to know what 37's state is and sends a switch binary get by way of node 18 on line 27000. This goes through very quickly on lines 27000, 27001, 27003, and 27005.

My trace must be missing a packet because on line 27009 we see the Switch Binary Report coming out of 37 by way of 18 but we never see the first packet from 37 to 18. Weirder still we see another duplicate packet on line 27011.

I'm guessing 27013 is another 37 binary report to 1 by way of 18 because 27014 is a third attempt by 18 to tell 1 about 37's binary report. Why isn't 1 responding to 18?

We see 37 give up on that route and try a new route through 9 to send the Switch Binary Report on 27016 and then suddenly three nodes report back to neighbors that they can't reach 1.

WTF, 1! What's going on? It didn't respond to 18, 150 or 42. (At this time two small rules ran.)

So, 37 tries to send the Switch Binary Report through a different route but this time 37 is kind of mad. He tries on 27029, 27034, 27035, and 27040, 27046, 27050, 27054, 27055, etc. There are probably even more. 37 is pissed. He ain't gonna be ignored. Nope. Lots of messages and lots of routes.

So, whose fault is this? Was 1 supposed to respond and 37 is supposed go nuts on all his neighbors if 1 doesn't respond? Or is 37 just totally being a whiny little b****?

In this story, here are the players:

1 - Hub
37 - 12721 GE Outlet (non-plus)
18 - 26933 GE Motion Dimmer (plus)
42 - 14291 GE Switch (plus)
9 - 26933 GE Motion Dimmer (plus)

Personification complete. The end.

image
image886×397 10.2 KB

2 Likes