I feel like maybe your rule allows for too much slosh. So add more restrictions or layers, maybe?
I actually do something similar with mode. My system is set up much differently than yours, although I'm only judging that by your one rule, so probably a completely inaccurate assumption.
HSM changes state according to mode. Home/Away/Night/Quiet time
I also use these states with my rules. It works quite nicely. A lot of my rules aren't based solely on state, so other restrictions can vary due to time of day. I'll explain the modes so that you understand them.
Home: that's obvious - at least one person home and no one in bed
Away: Again obvious - everyone is gone - hsm armed away
Quiet time - at least one person in bed. HSM Armed home (smoke/CO2)
Night - everyone in bed - hsm armed
Presence is managed with life360. it's been pretty accurate as of late. (They've fixed it). It is recommended that you use more than one presence sensor to avoid false reports, but know that the wifi may turn off on your phone when it's sleeping making that somewhat sketchy as well. Your choice as to what you want for triggers.
So my notifications work on mode
here is the rule to show me what mode my home is in, which determines how the house is armed.
I know it's not a straight up answer. But I wanted to respond to your question with how I deal with things in my environment. Maybe this will help you with that box we all seem to get stuck in every once in a while.
[EDIT]
I feel like maybe presence is hopping on you.
