One thing I know I did was to disable PMF on all of my WLANs.
It was a painful process, as I reverted my UDM back to v1.7.2 (from 1.8.3 at the time) which did resolve the HomeKit issue... However, my issue was more widespread than just remote access, IIRC, I believe I could not even access access things while on the LAN... 
I eventually followed all of the recommendations I found in the Ubiquiti forums for WiFi best practices, and then I upgraded to 1.8.3 again and things continued to work. So, it is hard to say exactly what setting may have 'fixed' the issue.
Have you upgraded your UniFi Controller Software recently? I am assuming you're using UniFi WiFi Access Points? Just a guess that the new Controller software might be implementing new default settings, as that what caused the UDM to change things in my home.
EDIT: Just found this old post...not sure if it will be helpful or not...
