While testing something else, I observed that my Generic Z-Wave Valve can open/close/on/off from the device page and operates fine from valve template on a dashboard. But within an RM 5.1 rule, closing the valve does nothing. However, turning it off does close it, and open/on also work.
The valve (that I'm physically watching) doesn't close until after the delay and the off is issued. See also that the valve device itself reports back that it was turned off, but nothing after the close.
Hey @HAL9000 I just noticed this same thing today. It's something to do with RM and the open close of the valve command. I somehow think they are backwards because my logs reports backs open when commanding close from RM. Try it as a switch instead (the valve will also be listed as a switch), it will work. That's what I had to do to get my rule to work properly. Maybe support can reproduce this and have a look at the issue @bobbyD
Just tested this with a virtual device, and it indeed appears Rule 5.1 is sending an open() command to the device regardless of which selection you make here: