Please give feedback, suggeations or collaborate with my LLM (aka "AI") related project

💬 Lounge
1

Hi. I've created a POC of my project and I am committed to go forward with it. However, I don't have time to develop it to its full potential and it would be nice having a few collaborators. So please shout out if you would like to help and have the skills required for doing so.

What is the problem I am trying to solve?
After years owning Hubitat, it is still always me who creates any automation. Although Hubitat doesn't require the user to have IT skills for creating automations, most of our wives won't bother creating any rule in Rule Machine.
In addition to that, there are oftentimes when my wife or myself would like to create some kind off "one off" automation. Something I could quickly create ad-hoc and have it running only once.
For example "When I arrive home, remind of X", or "This week, open the kitchen curtain every morning".
So my intention here is creating a very user friendly way for users to create regular and ephemeral automations.

What is the proposed solution:
I created a project (in Python, using Langchain libraries) that allows me to write a message to ChatGPT with any kind of instruction for my hubitat and that will do for me.

-Starting with trivial things, such as:
"Turn off all lights in the living room", "Send a message to everybody saying 'I will be home late'"

-Then reaching more interesting stuff related to the problem I described above:
-"When I arrive home, remind of X", or "This week, open the kitchen curtain every morning".

I expect to soon be able to handle requests like:
"Every Tuesday, send me a message when X has arrived"

Out of all the personal projects I've ever done, this is certainly the one my wife sees most value.

What I would like to ask:
I've come a long way implementing the LLM part, but there are lots of work for making this a real solution and not something sketchy running from my computer. For example:
-Create a simple android app where our families could write the messages to smart home. This app would basically allow the user to edit some settings and connect to the webservice I created.
-Implement a simple authentication method (a friend might help me here).

I would be happy to share the efforts with somebody else and later having more people using the solution. Right now I am keeping it to myself until the authentication and a few others things are implemented. Also, there is an inherent cost of running the solutions since it makes some calls to OpenAI API. (One of the things that took me time was designing the solution in a way that won't make it absurdly expensive with useless calls to OpenAI).

In the future, if we there would be interest, the solution could be expanded for allowing automations to be created based on data not related to the smart home (for example the weather, if you don't have that implemented in your Hubitat). Basically any data that could be fetched via API could have an integration with it. We could also store some data in a database (for example electricity consumption or user behaviour patterns, for example, the times of the day when the user said "good night") and provide different types of insight.
There are countless possibilities but I don't wanna focus on that now, my goal is getting the basics done and working well.

Any feedback on the idea? Anybody willing to help?

About me:
I work as Product Director in tech companies and have a background as IT consultant. So I know some programming but my expertise is much more on databases / analytics and Product management. This is part of the reason why I also would like to have partners on this initiative.

4 Likes
2

What do you think of the value proposition?

3

I have my reservations about how much Hubitat-specific results you may be able to get out of ChaptGPT, given the rapidly changing nature of the platform.

That said, my boss showed us an interesting feature of the paid version recently where he setup pre-canned conversations where he described what he would provide and what he wanted in response, taking in CSV like data and producing a tabular output (column descriptions in our case). So this style of setup in ChaptGPT could be worth looking at.

4

Re the Hubitat app side of things and adjusting settings, I might be able to include some stuff in my Tasker app, unless you would like to contain it inside your own app.

5

Don't worry about that part. Just using ChatGPT out of the box won't really be useful. What I have done is creating a RAG application (what is rag in ai - Google Search...)
so that I can teach the ChatGPT to interact with hubitat APIs and implement automation rules through that.
I have already managed to achieve that part.

For now I am not creating it as a Hubitat app just because it would be one more things for me to learn and to maintain.
At this point, the only thing that one has to do on Hubitat for using what I've built so far is setting MakerAPI to post the Events in the url of the webservice I am implementing.
I can let you know when I am able to accept multiple users and the application is more mature. Right now it is not really in production yet.

1 Like
6

I do believe the power of AI is the next step to Rule creation. I've been working on an app on and off for a while. Here's a couple of 'sneak peek' screenshots.

Basically the user can explain what they want to happen and then ChatGPT sends back some JSON. My app can then take that JSON and make an automation out of it.

chapgpt app 1
chapgpt app 11602×913 60.1 KB

chatgpt app 2
chatgpt app 21612×911 53.7 KB

2 Likes
7

That's cool. I would probably have tried that path as well if I knew how to rules from Rule machine could be written.
However, in your implementation we would still be lacking a way to enable other users to create the rules without having to access Hubitat admin page.
If there would be a way to simplify that part I think your solution would be less complex than mine, which is a good thing.
One down side of my approach is that the ephemeral rules (or any rule) created through my app would depend on internet connection in the household for work, since I don't actually create the rule in Rule machine.
One up side of my approach is that it enables for more flexibilization without making the user install new hubitat apps or drivers. For example, I can easily choose how each user will receive a notification, and have different settings if they are away from home or not.

1 Like
8

I should note that I use the term 'Rule' in general. It doesn't create a Rule Machine rule but one that my app generates, an offshoot of my Event Engine. :wink:

Now if we knew the exact JSON output of Rule Machine we could then 'import' it into Rule Machine. Now THAT would be cool. lol

2 Likes
9

How about forecasting battery life and lending some smarts to managing batteries in the absence of the devices themselves doing the job they should be doing in this realm. Despite all the "I haven't heard from xyz device in 24 hours" kinda warnings this dying battery thing seems to be a common point of dependability failure in HA.

While "helping the wife write rules" is an interesting challenge....I fear....it's gonna be all about the devices, their locations, and their batteries that'll be the epic downfall of man-kind's automations once we're not here to manage these environments we've build out.

1 Like
10

Has there been any progress with integrating with the hub from AI/LLM. I have started to dabble with LLM's recently and it seems some really interesting stuff could be possible.

1 Like
11

Here’s another thread that was created recently:

1 Like
12

Sorry for disappearing. I finally got to work on this project and I created something. I just posted some info here, asking for suggestions: