Group doesn't issue an off state?

I'm trying to set some notifications based on two groups of switches. I have two groups called lower level and upper level. These contain all of the switches on that level of my house. If a switch turns on it get a level on trigger in the log. But when the last switch goes off, I don't get a level off trigger in my log. I have the "use group device to indicate if any member is on" enabled. So why doesn't this trigger a group is off?