Does anyone have any examples or advice for writing unit tests for apps and drivers? I have to imagine that there's a decent way to do this because Groovy itself has a lot of built-in testing support from what I understand. I understand I'd have to mock out a lot of internal framework functions, but that's fine -- just want to unit test my code and logic.
Surely somebody has done this, right? I'd love to see some examples for inspiration.
@nate I'd love to talk about this. I'm working on a kit for the Intermatic PE653 Hot Tub controller. There's some existing code, but it's fairly complex, and I'd prefer to write with tests at my back.
I see a kit called Spock but I swore a holy vow to never again muck around with JAR files and I'm hoping there's something simpler.
Did you get anywhere?
@mike.maxwell @chuck.schwer @bravenel
I'm wondering whether either of you have any ideas here. The recent regression of zwave.multiInstanceV1.multiInstanceCmdEncap upped the ante for me.
I'd like to get development of DTHs under test. They're not classes, so I'm not entirely sure how to hook into them with eg Spock. I would make a class and test it, and use it from the DTH, but IIRC we can't create classes.
I imagine DTHs get instantiated as a class of some kind. I suspect that with some details of how that happens, I could write a wrapper for Spock that would allow development to be test-driven. This could enormously shorten development cycles - since you don't have to do everything with virtual devices or copy-paste-click!
Rather than stumble around in the call stack and figure this out, I'd love it if there was any light y'all could shed!
I would be happy to set up a simple script for running tests, provide a starter test suite for the DTH protocol, and write up a tutorial for using it that even beginners could follow.
As stated above you would need to mock out all the various functions that Hubitat provides in order to create a test harness for an app or driver.
Thanks @chuck.schwer! I was hoping to keep the HE runtime around so it could speak for itself, but if not, mocking wouldn't be too hard. . . . and really, beneficial results could be achieved just by testing eg the various action methods and what they return.
The biggest hurdle is that DTH files aren't Groovy classes. I imagine they get turned into ones by the HE runtime, though. Are there any details of that process, anywhere, or that you can share?
Unfortunately I can't share that information it is proprietary.
Sure, I understand. Do you have any suggestions for how to write test-driven code, then?
It looks like HE's class is com.hubitat.handler.BaseDeviceHandler, which dispatches via runDeviceMethod? A stripped-down version of the same loader should be trivial to produce, and would let developers write without copying and pasting code into the hub repeatedly to test it. Would that feature be possible to get, do you think?
Or, one could fairly trivially write a preprocessor that stripped out the metadata/etc and just inserted the methods into a new object, which would allow for testing.
What does HE do for automated testing?
Thanks;
Joshua
I know this is an old topic, but I would love to get something to help with unit testing apps/drivers. Manual testing is a frustrating and time-consuming experience.
It's not that bad, how many do you write?
I have ~6 drivers and one app right now. It's certainly not a large number. Generally, making a change in one of those drivers requires about an hour of manual testing (and it is easy to miss something). With automated unit testing I could reduce that time down to nearly zero, and have much more thorough testing to boot.
I'm pretty sure that all I really need is abstract classes for all of things that a driver/app interacts with. Groovy makes it pretty easy to mock classes out with just that.
Would I run afoul of anything if I wrote some code that scraped portions of the developer docs on https://docs.hubitat.com, and used that to create stubs of all the functions?
1 Like
In the meantime why not build an app to do your unit testing...
I do have something like that. Stubbed out all the APIs, but I also went further and perform some validation on all the metadata, inputs parameter names, command arguments, settings and so on.
You can basically point my framework to your .groovy script, it'll load it, validate the basics, create script object. Then you can call install(), uninstall() and other methods on that object to test it (all on your machine).
I have the framework in my private repo, and was planning in making it open source some time later. I "just" need to do some cleaning up and documentation.
Example (written with Spock framework):
class AppTemplateScriptTest extends
Specification
{
HubitatAppSandbox sandbox = new HubitatAppSandbox(new File("Scripts/New App Template.groovy"))
def "Basic validation"() {
expect:
sandbox.run() // Validates inputs and so on.
}
def "Installation succeeds and logs stuff"() {
given:
def log = Mock(Log)
AppExecutor api = Mock{ _ * getLog() >> log }
def script = sandbox.run(api: api, customizeScriptBeforeRun: { script ->
script.getMetaClass().ventDevices = ["S1", "S2"]
script.getMetaClass().numberOption = 123
})
when:
script.installed() // Run installed() callback and validate it.
then:
1 * log.debug("initialize")
1 * log.debug("ventDevices: " + ["S1", "S2"])
1 * log.debug("numberOption: 123")
1 * api.unschedule()
}
}
3 Likes
I would love to see this released as open source. Will definitely save me a lot of time with rolling my own.
Any idea what kind of timescale you're looking at for releasing the code?
I suppose I could make something available in a week or so.
2 Likes
Well, there it is: GitHub - biocomp/hubitat_ci: Unit testing framework for Hubitat scripts.
I wrote a tiny getting started doc there, let me know if you have trouble getting things going.
5 Likes
Thanks! That repo is far more complete than I was expecting. I was able to get some simple tests working, but I am running into an error when I try to use label or name in a device driver:
In [initialization] settings were read that are not registered inputs: [label, name]. These are registered inputs: [logEnable, txtEnable]. This is not allowed in strict mode (add Flags.AllowReadingNonInputSettings to allow this). Expression: readSettingsThatAreNotInputs. Values: readSettingsThatAreNotInputs = [label, name]
java.lang.AssertionError: In [initialization] settings were read that are not registered inputs: [label, name]. These are registered inputs: [logEnable, txtEnable]. This is not allowed in strict mode (add Flags.AllowReadingNonInputSettings to allow this). Expression: readSettingsThatAreNotInputs. Values: readSettingsThatAreNotInputs = [label, name]
at me.biocomp.hubitat_ci.validation.SettingsContainer.validateAfterPreferences(SettingsContainer.groovy:43)
at me.biocomp.hubitat_ci.device.HubitatDeviceScript.hubitatciValidateAfterMethodCall(HubitatDeviceScript.groovy:85)
at TestMyDevice.log function test(my-device-test.groovy:31)
I suspect this might just be a feature that's not implemented yet. I can probably take care of implementing it if you point me in the right direction.
Glad you were able to figure things out 
Do you want to create an issue in github and discuss it further there - I'm guessing this thread is not the best place for it?
I'm not actually too proficient with writing drivers. Could you explain what name and label are in this case? Are those some standard properties of a driver? I did not see them in DriverExecutor api (in which case they would have worked).
Starting point would be looking at getProperty() method which will be called for everything groovy considers to be 'accessing a property'. Then these accesses are recorded and validated against registered inputs (which are [logEnable, txtEnable] in your case) via SettingsContainer.
Alright, I'll open an issue on github. I didn't start there since not all everyone who posts hubitat related code is responsive to github issues.
1 Like
This is really cool, any plans to use this to run apps outside of the hub?
Thanks!
Running an app outside the hub is possible in theory, but will require implementing all the hub's internals, which I don't have the resources to do.
My framework assumes that you will mock the limited number of hub's APIs used by your code. This way I can avoid implementing everything (some things are implemented there, but very few).
