Can I ask what do you have the switch controlling.
I have had this behaviour with zen21s and zen26.
Zen21 - using three of them with an exhaust fans. One of them was giving me issues and I installed a snubber circuit (RC) across the fan to protect the switch from high speed transients when the motor is switching on or off.
Zen26 - many cases where the load on the switch was too small and the OFF current through the load was too low to keep the switch in a good state. You can test this scenario by temporarily using an incandescent bulb and seeing if it fixes your issue. A permanent fix here may simply be swapping non-dimmable bulbs for dimmable ones. Yes I know this seems silly but some non dimmable led bulbs will not allow any current to flow until a certain voltage is reached. This leaves the switch high and dry in the off state. An alternate solution is to use an Aeotec bypass (expensive) or load resistor.