Someone else was looking to do something similar and they just came up with a way to cancel the alert when it happens.