[RELEASE] Hubitat Package Manager (HPM) -- HubitatCommunity

We recently got my new repo hooked up, but HPM doesn't show the Ring Keypad Driver package using Search by Keyword (fuzzy) Ring or Keypad. The package does show up for all other methods, including Keyword (fast), Tags, and From URL.

What do I need to add/fix to make keyword (fuzzy) work ? I looked at hpm docs and browsed some other folks' repos, but I'm not seeing it. I see it's happening to some other packages too, like Iris v1 Keypad.

"Fuzzy" uses Dominic's Azure based search. When he announced the first time that he was moving to HA, I created the "Fast" replacement because I assumed he'd discontinue Azure. He didn't and even said he wouldn't, but "out of sight, out of mind" may occur and he wouldn't be checking it. Like "Fast" it automatically reads in all the manifests and builds a search DB... I know that I've set "Fast" to rebuilt every 20 mins. It's possible Azure gets "stuck" and needs a manual intervention. If so, I can't imagine he's checking often anymore.

2 Likes

I spent some time looking at each of the two issues I mentioned:

  • What to do about message wording that can be interpreted alarmingly.
  • What to do about missing/removed Packages.

I've gone through the code looking for Error level messages and how best to interpret them. Not surprising to me, Dominic's code is very good at ignoring/skipping them. He just chose "alarming" words I think. Looking at that #300 above, all of those are ignored. HPM just keeps on working, as I mentioned. As yet unreleased, but I've 'downgraded' them to "Info" level and reducing the intensity of the words.

These "errors" are problems that the Package Developer introduced and don't cause HPM to stop or fail. HPM just ignores them. Admittedly, the Manifest flaw can have an effect elsewhere, but for these examples from MatchUp, they have no direct effect. By that I mean: if the developer has intentionally removed the package or manifest, then HPM can no longer find updates, install, or repair those particular Packages. You should treat them as information and choose how you wish to handle it. Contact the Developer certainly and see if it's intentional.

I've also looked at how to assist with "Owner Migration" of a Package. I'm testing for unintended consequences, but I have made some progress:

info  A repository was removed, [https://www.hubitatcommunity.com/hpm/communityrepo.json]
debug missingRepositories: [[name:Community Code, location:https://www.hubitatcommunity.com/hpm/communityrepo.json]]
info  A new repository was added, http://192.168.7.42/communityrepo.json
info  Refreshing repository list

info  A new repository was added, https://www.hubitatcommunity.com/hpm/communityrepo.json
info  Refreshing repository list
info  Matching up packages

I created an alternate Manifest system that I can edit viciously and not affect anyone. I killed an entire manifest from the fake Master. Then put it back. Dominic's original code correctly discovers the new manifest. (bottom 3 lines)

Changing the location (URL) means it's an entirely new manifest and the original code found the new URL, but my experimental code detected the missing manifest and removed it.. effectively transferring the repo.

As I said, detecting unintended consequences is going to consume a lot of testing time. I don't want this to be ONLY a tease.. but I'm sure this won't be ready for release quickly. :smiley:

10 Likes

image

v1.8.8     check for Null for Invalid Category & Tags
             log.error changed to log.info where the event was handled.
             detect/delete missing Repositories.
12 Likes

@csteele In case you wondered, YES we really appreciate your work on keeping HPM humming. It really is an essential part of making HE a viable platform. Without community drivers/apps there would be big holes in functionality. And without HPM we'd all be chasing our tails staying up to date and figuring out annoying little details of each install. Thank you!

16 Likes

I suspect any existing users of HPM reading this won't benefit from this option, but I created a Bundle for HPM and it's out on GitHub.

https://bit.ly/3VfykH9

which is a shortened version of:

https://github.com/HubitatCommunity/hubitatpackagemanager/raw/main/HubitatPackageManager_Bundle.zip

A Bundle solves a very few issues, but for NEW Users, I suspect they are large. I'm sure you know that Hubitat splits code three ways, into Apps, Drivers and Libraries. New users of Hubitat shouldn't be expected to comprehend the differences and when they don't, given it's 33% guessable success rate, getting it wrong adds frustration at the worst possible time. For HPM, that's ALL that a Bundle cures... dropping it into the correct code bucket. The clicks, the copy/paste of a URL is all the same as an Import, just save the esoteric "this isn't driver code" message. :smiley:

I suggest, for anyone wanting to assist a new user, they offer these instructions:

Click Bundles:
Screenshot 2022-12-20 at 12.00.43 PM

Click Import ZIP:
Screenshot 2022-12-20 at 12.00.54 PM

Then click "Download from a URL" and paste the URL above into the field. Click Import:
Screenshot 2022-12-20 at 12.01.14 PM

Wait a few moments and "Bundle Imported" appears.
Screenshot 2022-12-20 at 12.04.05 PM

From there, it's just clicking Apps:
Screenshot 2022-12-20 at 12.05.08 PM

then click Add User App:
Screenshot 2022-12-20 at 12.05.14 PM

and Pick:
Screenshot 2022-12-20 at 12.06.54 PM

Finally, click Hubitat Package Manager in the list of Apps and HPM fun begins. :smiley:

Note that the Bundle does not HAVE to be up-to-date. It's intended to streamline, and thereby reduce frustrations for new users. It might, in fact, have some benefit by being behind by displaying "Updates Available" rather soon after Install:
Screenshot 2022-12-20 at 12.11.21 PM

There's sure to be some benefit to clicking Update within HPM and seeing it work. :smiley:

20 Likes

Hello all - After running webCore on ST for many years, I'm finally moving over to Hubitat since webCore on ST will stop functioning on 12/31/22, it seems? Never a great time to have to go through a new learning process with so many other things going on, but I am looking forward to having local webCore execution, as ST has really gotten slow. But I'm sure I don't need to tell you guys this!

Anyway, in looking over this thread for how to get started on Hubitat with webCore, I'm not exactly sure the best way to proceed... The first post in this thread, which seems to be the latest instructions, is from May. So I fear this may get me started on an older version, and I'm not sure how easy it is to upgrade from there.

Conversely, this latest info from csteele just a week ago seems to make things easy. But I wonder if I follow these steps if it will somehow lock me in to a more rigid upgrade/update process in the future, or have some other downside?

Thanks all!

Hubitat Package Manager is at it's root, an alternative to all the clicking needed to get Community created code into the Hubitat Hub. There are several pitfalls with manually adding code.. putting Apps into Apps Code, and Drivers into Driver Code, seems to be the first challenge :slight_smile: Next, Hubitat's GUI allows us to either paste all the code, or Import via a URL. If we choose to paste the code, which is usually stored out on GitHub, then navigating Github and actually copying all of the code, is the second challenge. It's easy enough once you've visited Github a time or two, knowing to click Raw. Importing has the same copy paste (grab all the letters) challenge but is a little bit harder to get wrong.

Hubitat Package Manager (HPM) has simplified all that.

Layered on to those advantages is HPM's detection of updated code.
Screenshot 2022-12-26 at 9.43.03 AM

That's all it does, tells you there are updates, you don't have to use it. It helps with repairing and adding optional components to installed Packages.

I think I've made it as simple as possible to Import HPM for the first time using the Bundle. I think there are fewer challenges, especially using the bitly URL :slight_smile: https://bit.ly/3VfykH9

as detailed above:

If there ARE Updates available, HPM offers to individually update:

image

4 Likes

Fantastic. This looks like a wonderful update. I'm curious if anyone has used this yet. I would imagine a fair number of people are moving over from ST to Hubitat to save their webCore pistons from going under. But then again, perhaps I'm last to move over. :slight_smile: Anyway I'm very grateful for the work that's been done to keep this easy as possible.

This is such a massive advantage. I have over 40 packages (apps or drivers, or both) installed from HPM. The amount of work saved compared to manually updating them is enormous. I have probably only 6 or 7 things installed that are not in HPM, and it's always an annoyance when one of those needs an update.

5 Likes

I've created a Video that shows the step by step on an Empty hub. I wiped one of my hubs and show the Installation of Hubitat Package Manager and then using it for the first and second time.

HPM Installation - First Time Users

In order to same some space, the video moves at a rapid pace. Use the browser's speed control OR the Pause button on the playback.

16 Likes

Is it possible to have HPM work over HTTPS only connections using non-public certs/CA? I have my own CA and use my own certificate for hubitat and when I have HTTPS only set HPM doesn't work fully (For example, says it's installing a package but nothing actually installs but I also don't see errors either)

HTTPS Only Set
image

Hubitat App Code

Hubitat Drivers Code

Question for developers that use HPM -- what is the best way to handle an app that is relevant between two unrelated device integrations?


I have two different camera system integrations. They each support useful functionality (motion, doorbell buttons, etc) on their own.

I have an app that is useful (and about to be necessary) for various use-cases with snapshots captured using virtual devices from each integration.

I'd like to single-source the app and just reference the same location in both HPM package manifests. But I think that HPM will happily uninstall and install the actual source in Apps Code, even if HPM tracks internally to itself that two unrelated copies of it are installed.

Has anyone in the community handled something like this before? Any thoughts based on your knowledge of the HPM code, @csteele?

I think you're looking at Optional installs:

  "primary": true,
  "required": true

  "primary": false,
  "required": true

  "primary": false,
  "required": false

Each component in the Package Manifest would have one of those pairs. Then during install, the option menu.

Screenshot 2023-01-07 at 9.58.44 AM

In that Package I selected the "main" app to be required and primary, then a component to be required and not primary, which tells it the order to do the install. The final component (dim) is completely optional.

1 Like

Interesting, I didn't know about primary before. Can you explain more about what this does?

I use required extensively. For example, I have several integrations with drivers (required) and apps (optional, with additional functionality).

What I have here is different, in that I want to have the same app be optional for two different integrations, which may be installed on the same hub. So, I want to make sure that either can add it, but neither can remove it out from under the other.

Does primary help with this? Or is there another way to do this? I thought about using bundles, because the source files get left even if the bundle is removed, but that felt like sort of a hack.

1 Like

I tried this out by mocking up two packages. The both included an unrelated "required" driver and the same "optional" app with the same "id".

When I installed the packages, it installed two copies of the app. And I could remove each package and it would remove its respective copy of the app.

This helps with the concern about one pulling the rug out from another by deleting the source. But it creates a confusing situation where there are two instances of the app installed and usable, and I really want one instance of the app that can reference instances of different drivers/devices.

It looks like my solution is going to be to have the shared app as its own standalone package and to recommend it as a separate install in the readme of each driver package that it supports.

I'm getting closer to understanding your intent, but I'm not there yet.

My interpretation is that you have one App and a 'collection' (one or more) companions that fall into two targets... You have Package 1 that contains the App plus companions A, B and C, while Package 2 contains the App plus companions D, E and F

You want one Manifest that will read the users mind and install either Package 1 or Package 2. I can see it working with two manifests.. a Package 1 and Package 2, each with their own list of components and with overlap on the App, which is common.

I can see it working with a single manifest that has everything and the companions are all optional. Naming the various companions would then guide the user to picking the ones needed.

The App would be

  "primary": true,
  "required": true

Then all the companions would be:

  "primary": false,
  "required": false

"primary" defaults to false so it's redundant. "required": false means that an option menu is displayed and the user can pick one or more.

Screenshot 2023-01-07 at 3.12.29 PM

I can't think how it would hurt to install all the companions. But I know OCD can be a factor :smiley:
If the App and all the companions are added, does the App know what to do? Or is mind reading required also? You're hoping to install just the pieces that would clue the app into which form it takes?

As I said, I'm not convinced I have a grasp on your intent yet, but this is my best guess from the clues. :smiley: HPM doesn't have If-Then-Else logic in the Manifests.

My mind map is the opposite of yours in terms of which is the companion to which. I'll describe it in more specific terms, and hopefully that helps:

Package A is for camera manufacturer A. It has a parent driver and some child drivers for various types of hardware. The child drivers have a command that stores snapshots in a particular format.

Package B is for camera manufacturer A. It has a parent app and a child driver for the hardware from that manufacturer. The child driver has a command that stores snapshots in the same format.

The app I'm trying to figure HPM support for knows the same snapshot format and serves it up via an HTTP endpoint (for dashboards, etc) and can also send it in a Pushover notification. In the app, you can select devices of both types (A and B), and then the app does its thing.


Having the shared app installed by either A or B is straightfoward -- I'd mark it as an optional "apps" entry in the manifest.

It's unlikely, though not impossible, that a user will have hardware A and B. So installing both all the time would be very weird. But the possibility of them both existing is the problem I am trying to solve. I have both on my system now, and it is affected by the problem I described in my previous post -- two instances of the Apps Code installed, etc.

I am seeing something odd [BUG?] with the MODIFY option. Even when unchecked in the list, HPM still tells me that it's going to uninstall the package. I am not sure if it will indeed uninstall it as I didn't continue.

On the Modify window, you should check the things that you want to have stay installed. Unchecking them indicates they should be uninstalled, sort of like having them unchecked during the initial Install step. You can also check new things during Modify if you want to add them.

1 Like