One approach is to make a device like the one linked below (and there are many examples of such devices on the Hubitat community).
I have two of these that have worked well for ~3+ years.
With this kind of device, the contact will appear "closed" when there is power, and "open" when there is no power.
I haven't had to change the batteries since I built them.
TheSmartestHouse.com also sells a Power Outage Detection Kit, that they are currently out of, but it works on a similar principle.