Happy to report the first two are still working fine, so of course had to add another one.
After discovery it was doing the "three event" thing, multiple reset/re-pairings later it's working properly (1 push = 1 log entry) as well.
Unfortunately I didn't really keep track of the process (if there is one) to get it paired correctly.
From what I remember:
It was more than five but less than thirty times I pushed the "link" button four times to reset the button.
Deleted and re-discovered the button once. Initially the "Device Data: Details" section had no in/out clusters information until after the device was deleted and re-discovered from HE.