What terminals did you use for everything? Did you only use the outputs (large terminals), or did you wire your wall button into the Zen17 too (small terminals)?
Have to, probably not. But it will be much easier if you use that Smartthings or another similar sensor on the door. That will become apparent in a later section below.
So what did set the main device settings to? Screenshot of that page?
Also, you are going to have to set this up for garage door mode. There are ways to press the buttons on the Zen 17 to do so, but it was easier for me to use the Super basic Z-Wave parameter tool to set everything needed and check all the settings. The short version is you install this tool, and temporarily change the Zen17 driver (device type) to the Basic Zwave Tool. Open the logs in a new window, and back in the Zen device you can query the parameters to see how they are set. The output will be in logs. You can then set parameters as needed to become a momentary relay.
The parameters you need to check are listed in the manual, but you likely should double check at least parameters 6-9 and 15-18. I checked every parameter in the manual just to be sure it was set correctly. You basically want to be sure you have the auto-off timer for 1 second, and time units in seconds, and no auto-on timers set for the relay you use. Hopefully this will be easier in an upcoming Hubitat firmware, these parameters should have been exposed in the UI, but somehow they were omitted. https://cdn.shopify.com/s/files/1/0218/7704/files/zooz-700-series-z-wave-plus-universal-relay-zen17-ver1.0-manual.pdf
The last confusing part was that I couldn't get a dashboard button to control things. After digging a bit, there are a couple apps that people have written to be garage door apps that combine a tilt sensor and the relay. I ended up using kahn-hubitat's [Release] Garage Door Opener App - LGK Virtual Garage Door which is also in Hubitat Package Manager if you use that. It was very simple to set up, and gave me my garage (control) dashboard button back.