You could use a physical button. We don't have a UI method to do this, other than if the app were opened, and a "switch" turned on. There is an example of this in Hubitat Safety Monitor, where the Cancel Alerts switch is used for a similar purpose. What happens in that UI, is that if you turn on the switch, it does cancel alerts, and immediately turns off again,
We will look into adding a "button" ui in apps as well.
