Well that's what's happening right now since they're not supported 
4 Likes
Might be easier to support the Bundles …
1 Like
Love HPM! For your consideration, my idea on how to integrate libraries into HPM:
- In the packageManifest.json files, you currently have "apps" and "drivers" sections. Add a new "libraries" section. I haven't played with it enough yet, but I assume the libraries would need to be installed/updated before apps and drivers. (Since I believe under the covers Hubitat is doing a find-and-replace with the #include statements.) That right there would allow sharing of library code between apps and drivers that are in the same package. But to share libraries between packages, we still need:
- A way to indicate that one package depends on another, and requires it to be installed first. I think you could do it by adding a section to the manifest where you list out the URLs to any other manifests that you depend on.
In my case, I currently have 11 packages registered with HPM. They all have duplicated code. I would extract that code into a 12th package, which only links to groovy files in its new "libraries" section. No apps or drivers. I'd call the new package "Joel Wetzel Libraries" or something. Then my original 11 packages would now all have a dependency on my library package.
I don't think you need to do namespace mangling. I think URLs to manifests were good enough of a primary-key and versioning system for your repository.json, and they'll be good enough for indicating dependencies. (URLs can indicate a specific revision of the file, if it's in Github or similar.)
Note that what I described above would allow people to reference the packages of other developers. But every package management solution I've worked with (npm, nuget, etc) all require you to explicitly reference a version, and/or do the work of keeping your package up-to-date with the dependencies you declare. There's no getting around that, and I don't think it's HPM's problem to solve. It's up to us developers:
- If I tell people that they're free to my your library in their own code, it's my responsibility to minimize breaking changes, or else I'm a jerk.
- If I reference someone else's library package that they don't advertise as usable by others, it's my fault if their changes break me.
Final thoughts: Edge cases. What if multiple packages all reference the same library package. Shouldn't be a problem, as long as they all reference the current (or same) version. You'll only have to have the library installed once. What if they reference different versions? That's the messy case, and namespace mangling might be the only solution. I still don't think it's a good idea. I'd rather have an HPM that just says "HE only allows one version of a library to be installed, so keep your code up-to-date." KISS principle.
5 Likes
All of this sounds like it is converging toward the Unix “make” command.
1 Like
In the end, doesn't everything? 
1 Like
Eh, for now I plan to just use libraries in the background, and only post the non-library/flat version on HPM.
Yes it is one more step to post them, but takes me out of this versioning discussion altogether until the dust settles.
Of course I have fewer drivers/apps I maintain than a lot of others do.
1 Like
It seems that this is how the code came to be exposed in the Production release. Internally Hubitat was using this for themselves.
I thought to do exactly the same, just use it behind the scenes and publish only the 'flat' version. Have you looked at the resulting code? It's unreadable with a comment per line. Easy to strip it out though. But that's an additional step again.
2 Likes
I am ok if it is unreadable by end users. 
All kidding aside, I only looked at it briefly and noticed the same thing. I haven't thought through how big of a deal I think that is to be honest. So maybe that won't work out as I had planned.
1 Like
Me either beyond the flash of a thought "oh boy, an additional grep/ack/sed post process step is needed."
1 Like
But they make writing drivers with shared code so much easier! 
(They really do--almost all of my Hue Bridge device drivers build progressively on less-featured drivers, so switching to libraries cut down on both the annoyance of copy/paste between drivers as well as likely occasional errors resulting from the same.) I think the main issue is that they really probably should not be shared among different developers, say as as "community" libraries (even though this is something I know some people have posited), and possibly not even between different projects from the same developer, at least not without following suggestions staff have made like adding a version to the namespace or name (where presumably this would change any time you risk breaking compatibility).
On some level, it seems like these would easily fit into HPM in a very similar way to existing apps and drivers--just specify them in your manifest. But thinking about it more, since libraries can affect other code, even developers who re-use their own libraries in different projects would need to be careful (see the above and the "versioning" suggestion), and I understand the concern with HPM for one package having the ability to affect another. But we can't control what people do, and this is really already possible on some level with any custom code (parent/child apps look for each other by name and namespace, too, though no one in their right mind would probably try to re-use someone else's parent or child app or create a manifest for fun that refers to it anyway).
There's also a new "bundles" feature that might be worth considering, and I don't think there's anything special there to prevent toe-stepping, either--just a general understanding that if you have something with a name and namespace that matches, it will overwrite it (whether it's a library or app/driver code). Part of me thinks it would be neat if we could replace large parts of the HPM manifest with something that refers to these instead, as they contain all information needed for installation and installation order (including libraries, apps, and drivers), so HPM (or the mainfest creator) wouldn't have to worry about it. The same concerns with libraries remain, though, and it perhaps opens up even more concern with apps and drivers (which for the most part don't mind if you have two with identical names/namespaces, parent/child-finding aside; you'll just have trouble finding the right one to install! -- but no GUID-matching like HPM helps with). A separate issue, but not totally unrelated and something I've been wondering about myself... 
1 Like
Interesting thoughts. So maybe HPM should allow us have a library as part of a package manifest, but not get involved in package dependencies. If I want to re-use the same library across multiple of my packages, I can have the library groovy code in just one git repo, and have all of the different package manifests reference that same URL.
4 Likes
I like the idea of being able to #include code, but it seems some comments are being added automatically to every line and this breaks all multiline strings.
This is what happens if you have multiline comments in a library and then #include them in your device:
Library looked like this:
I think a potentially better idea would be to have a comment before and after the included code, not on every line.
2 Likes
I bekieve the identifier after each line helps in creating error messages that point to the actual code line in th library. Using header information would make single line error identification more difficult.
1 Like
Libraries from my testing don't permit multi-line quotes...
Agreed, did not think of that.
Just something one needs to keep in mind when implementing things. Took me a bit to understand why my http request was failing (the request body was messed up due to me using a multiline string to define it and this was inside a library).
3 Likes
I have a bunch of drivers that would hugely benefit from the library (mostly from maintenance). If I use the library, does it reduce the amount of resources that drivers / apps use? Or does it just get "copied" into each instance on the Hubitat so no savings there?
No savings, just makes source code management easier.
1 Like
Thanks, still a benefit. Every time I go back into a driver I did, I find something that wasn't updated from other ones.
@gopher.ny have you written anything else up? The link to documentation doesn't appear to be populated. The discussion in this topic may be relevant but turns to version control and compatibility so quickly and abruptly that this topic isn't sufficient. If you question why this discussion topic isn't sufficient I'd start with a concise description of what libraries code is used for and how it could be used - or just put a disclaimer if you don't know what libraries code is used for or how it could be used then you shouldn't even bother to be asking the question.
Then at least you would let guys know that if they don't know what something is or how it's implemented in HE then these aren't the droids your looking for or perhaps that you shouldn't be looking for.
Can you point me to this link (both where it is here and what it points to)? Just wondering if there is something that could be updated. Thanks!
3 Likes
