I've seen it too, I put up a post a few days ago.
If the on bit of the rule isn't triggered the the off bit doesn't run.
I was under the impression that this wasn't meant to happen in motion lighting only simple automation (formally simple lighting)
As of yet I've had no feedback