It seems unlikely that an app could be generating events on behalf of a physical device, but if that's your concern there's one way to tell for sure: disable the app (or remove it), then see if the problem continues. If you want a similar automation in the meantime, you should be able to recreate this automation in Rule Machine (rule/condition = sensor open; actions for true = send notification with a delay on that action (or use a separate delay action before this) for the appropriate amount of minutes with "cancel on truth change" set.
I've had odd problems with the Notifications app before, but I've never used it for this purpose. In my case, I had it set to notify on use of specific lock codes, but it ended up notifying me regardless of which code was used (even reporting the correct code in the notification text). Your problem seems quite different, but I was able to work around it by re-creating this in RM. I haven't tried since then to see if the problem has been fixed. Can't hurt to see here, too.
