[BETA] ChatGPT Driver for better decision making

[Xagon]

Announcing: ChatGPT Driver for Hubitat - Beta!

Hello All!

I'm new here, though a long time lurker, user and developer, but not so much contributor! However, I'm excited to share the beta release of a driver I've been using for a while now. ChatGPT. This driver harnesses the power of OpenAI's GPT models, bringing advanced and contextual responses directly into the Hubitat system.

Features:

• Direct communication with OpenAI GPT models from Hubitat.
• Intelligent responses to user questions.
• Supports both general inquiries and true/false styled questions.
• Customizable settings to control OpenAI model behavior.

Beta Version Highlights:

• Interactive Responses: Ask your Hubitat anything, and get insightful answers powered by GPT.
• True/False with Detail Mode: Specific mode for boolean queries with added explanations.
• User-Friendly Configuration: Easily tweak the model's behavior to your liking.

How to Get Started:

1. Signup for an OpenAI API key at (OpenAI Platform)
2. Add the driver to your Drivers Code section.
3. Configure the driver with your OpenAI API Key and preferred settings.
4. Create a virtual device and use the ChatGPT Driver.
5. Start asking questions!

Beta driver is here: GitHub - xagon0/chatgpt-for-hubitat: A ChatGPT driver for Hubitat

Feedback:

Since this is a beta release, I'm eager to get your feedback. Bugs, features, or just sharing your experience.

I've had quite the enjoyment in building rules around it's answers, and have some plans to further expand the driver, maybe with custom function support, or similar.

But why?!:

Lots of people asked me why, my initial use case was for better control of various heating devices in my house. It's easy to say - oh the room is cold, turn on the heat. However, now I can chain together a few variables and say "It's {9:00pm}, the bedroom is {18c} and the entryway is {20c}. It's {expected to snow tonight, with an overnight low of -28}. The preference of the bedroom temperature is {16} to {19}, and entryway is {18} to {21}. Should the heat be turned on, and at what ideal temperature?"

I get a response of:
responseAnswer: The heat could be turned on in the bedroom to maintain a temperature within the preferred range. An ideal temperature would be around 18c, as it's within the accepted comfort range and closer to the lower limit to conserve energy while considering the significantly cold overnight forecast.
responseValue: true

Another integration I've enjoyed is just passing it the state of the house, and having a nice summary (that varies) each evening, played over EchoSpeaks.

System Content: You are a home automation system. Produce a concise narration to be played over the home based on the following information. Include only items you feel are relevant.

User Question:
Current time: 9:07PM Front Door: Last Locked at {8:45PM} Garage Door: Unlocked by Sam at {7:42AM} Back Door: Last Locked at {Yesterday at 3:02PM} Current Weather: {-9c, Cloudy. Wind northwest 20 km/h gusting to 40 becoming light near noon. High plus 3. UV index 2 or low. Snowing} Overnight Weather: {Partly cloudy. Wind up to 15 km/h. Low minus 7. Wind chill minus 10 overnight.} Security System: {Armed, No Faults} Lights: {Basement Lights: On, Living Room Lights: On} Thermostats: {House: 18C, Garage: 12C, Shower Floor: Off} Emergency Alerts: {None} Schedule Reminders: {None} Local News Summary: {State of emergency was lifted. No longer a need to conserve water.} ChatGPT Context notes: {reminders for plugging in cars below -10, or if I'm going to have to snowblow in the morning, maybe remind me to check if it's fueled up? You know, the standard stuff.}

responseAnswer: Good evening. The time is 9:07PM and the security system is fully armed with no faults. All doors are locked, except for the garage door that was last unlocked by Sam this morning. Please remember to check the status of the garage door. The current weather outside is -9C degrees and it's snowing, with winds gusting up to 40 km/h. You can expect it to be partly cloudy overnight with a low of minus 7C degrees. Inside, the house thermostat is set to 18C degrees while the garage is at 12C degrees. The basement and living room lights are currently on. Regarding your water consumption, the state of emergency has been lifted, so there's no longer a need to conserve water. Given the current temperature, consider plugging in your cars. Also, check if your snowblower is fueled up in case you need it tomorrow morning. There are no major emergency alerts or reminders for tonight. Enjoy your evening.

So...:

Ultimately, I hope I'm not the only person that finds it useful, and I hope I didn't miss a similar driver elsewhere!

[JumpJump]

Question 1: At the moment the driver passes all of the information to ChatGPT and ChatGPT is mostly just turning it into natural language that you then convert to locally to audio?

Question 2: Was any of the code generated by ChatGPT?

[Xagon]

Yeah, so for example number 2, the driver can pass information to ChatGPT for natural language processing. However for example number 1, it has the ability to provide context-aware insights, and could make complex decision-based on the provided inputs. A portion of the above (like emergency alerts, and schedule reminders) are actually from a first run with the same driver, passing along a giant collection of data from various news sources, websites, outlook schedule, etc. So while that is mostly language processing, it's making 'decisions' based on what seems relevant.

With the expected release of image upload into the API in the coming days, I have plans to implement the ability to send security captures, etc. to the API for additional insight. (Like better awareness of packages being delivered, or what the weather is actually like locally).

On Q2: The initial outline of the driver was created with ChatGPT, however at this point not much more than the method names remain. ChatGPT was consulted and used to guide and improve on efficiency. However, core logic of this release is my own.

Personal note: C# is my daily language, so learning Groovy over the past year plus has actually almost entirely been through ChatGPT, and trial and error.

[Equis]

Are both gpt-3.5-turbo and gpt-4 for ChatGPT Plus plans only? Can this be used with the free plan?

Edited to add: It looks like free API usage expires after a certain time and my account is too old to use the API for free. I am unable to submit a request via the Playground. Bummer!

[Xagon]

Yes, good point - that I definitely failed to mention! This does require an active OpenAI API account with an associated credit card. (Separate from ChatGPT Plus). Currently GPT-4 is billed at $0.03/1k tokens and GPT-3.5-Turbo at $0.0015/1k tokens.

For new users of the API there used to be a free 3-month trial. I'm guessing that may not be available any more.

This driver is averaging me around $11 per month against GPT-4. It would be less than $1 against GPT-3.5. Though I find the reasoning with multiple variables on 3.5 isn't good enough for my preference.

[JustinL]

Cool idea. Not sure if or how I’ll use it but I like the integration and am sure it will become more and more useful.

[Kulfsson]

I just had to try this, so I signed up for some tokens and installed it. I managed to get your examples to work, but not sure how I would continue to make it automatic. I assume you would use RM to push the system context?

On that subject, in either the driver or your explanation there is a small typo, content vs. Context.

[Xagon]

At present, I've chained most of my usages with Rule Machine. While I do have hope to integrate direct custom functions, I'm unsure of what that will look like in the future. Most likely still using RM to trigger those calls.

I've found that the [true] [false] responses tend to give me enough control over most items, but I've got number array responses working as well - so that may be coming soon.

This is an early snip/example of how I used RM:

And noted! I'll see about clarifying that across the project. The API uses the term "content" for the field passed along with a "role" name. However, when the role is set as system it's meant to provide "context" to the request.

[maxnet21]

Hi @Xagon ! I was just thinking about something similar to your driver when I found this thread!
Good job starting this up! I've been studying LLM and learning how to implement my own bots using ChatGPT Service. But for now my first goal with Hubitat is using ChatGPT for running a few times a day to act as a type of concierge, proactively fixing things -or at least notifying me, to begin with.
For example, if I leave home and forget the heating turned on. I will try to train the AI to realise this is not ideal so it would notify me, suggesting to turn it off. (Before people ask: I don't expect this experiment to add lots of value today, because I have a bunch of rules that serve me well, but I believe this kind of service could be useful in more dynamic environments where the implemented rules often end up being to rigid or creating confusion to some house members.

I'd like to ask you: I didn't get well the print of the RM you shared just above, could you please share more details on how do you set this "Replace (Z)..." ?
Tks!

[Xagon]

Certainly @maxnet21! Let me clarify how the "Replace (Z)..." operation is set up in my rule. It appears the initial snippet I shared was a bit incomplete, missing some crucial steps about variables and their assignment. Below is a more comprehensive explanation along with a detailed screenshot to illustrate the entire process. (This was just an example, so in actuality I'm also populating the outdoorTemp and baseline values from other locations.)

In this rule, I'm manipulating three main variables:

• baseRequest: The template for the request string that I'm going to modify.
• request: The final request string that's updated with the actual temperature values.
• currentTemp: A variable to store the current temperature from the thermostat.

The sequence of actions in the rule is as follows:

1. I initiate the request variable by setting it to the value of baseRequest using the 'Set Variable' action.
2. The currentTemp variable is updated with the actual temperature using the 'Set Variable'/'sensor value'/'Temperature' action.
3. I use the 'Set Variable'/'String'/'Replace' actions to substitute the %currentTemp% placeholder in the request variable with the actual temperature value. (In reality I use the hub variables for most things I access regularly.)
4. After updating the request string, I feed its value into the askTrueFalseWithDetail() function.

I've also included an updated screenshot showing the full configuration of the rule for clarity. I hope this better explains how this example was setup in Rule Machine.

[maxnet21]

Tks!
One thing I've been thinking is that it could be much more practical if we could instead have an Hubitat App for ChatGPT to act in combination to your driver. In the app we would be able to select the devices we want to expose to ChatGPT (maybe also select which properties to share) then everytime we wanted ChatGPT to make a decision we would pass as part of the prompt this device list (maybe JSON format, it seem to work well with ChatGPT prompts) . With that information in hand we could then have a bunch of different questions that ChatGPT could decide on by analysing the status of the devices (e.g. is there nobody home and there are lights on?; is it raining soon and the windows are open? etc).
What do you think?

[PunchCardPgmr]

Good brainstorming on how to evolve the OP's original concept.

The "ah, but this relies on the Cloud" folk may complain but the flexibility of HE tends to invite this kind of creativity. The AI door was bound to be opened by somebody someday.

No matter the cautions given, Hubitat is going to be used/applied in ways that could cause a lot of havoc ...even if not explicitly "Security Applications" (the use case where warnings are often given).

It's almost like you'd want some deliberate process requirement of a backup decision-making path or manner of default/fail-safe and acceptable variable ranges that you force into the Rule making.

[michael.l.nelson]

ChatGPT-2 can run in an excel spreadsheet. It's the training that needs the heavy processing. Once a model is built it can run on much lighter hardware. This concept can be done locally and run on a server in your home.

Edge AI is the next frontier, meaning executing the model on small local appliances, close to the data source. Look up Intel SceneScape and watch those demos. The minimum hardware to run SceneScape is an i5 with 8GB of RAM.

By the time the C10 hub ships, it will have AI. (source- I made it up.)

[Xagon]

I've thought extensively about these questions myself, along with trying a few various ideas. I'm often finding that GPT can take over most rules, but it often takes much more input to have a similar expected result.

In testing various approaches, the best solution I've come up with so far is a custom Hubitat app, along with an external server app that can interface back and forth with OpenAPI while exposing an endpoint. The reason I went with this approach was so that I could keep context (chat history) alive for all previous and future calls - along with the ability to use the API's function calling. (To actually directly action the request, or request more information).

Generally speaking this does result in producing rules that operate similar to Rule Machine, but can be a bit more simple to implement.

ie: "If the kitchen light gets turned on after 8:00pm, if there is no activity, turn it off after 15 minutes."

In RM that might be a few staggered steps that check for activity with light switches, motion, camera, etc, with GPT it's as simple as that request. However, GPT once in a while does decide it wants to just do it's own thing, which I'm working to resolve. I've found myself standing in the dark more then a few times.

For example 1, generally the logic is sound, but the requirement to make these requests on a regular interval really defeats the purpose of using GPT. It's less precise then a rule with a timer, and has potential for failure. It also fails to actually check if the light is on or off before making any decisions. (Plus, is this the best approach GPT? If the light turned on before 8:00, shouldn't the timer start at 7:22 + 15, or at minimum 8:00+15, instead of 8:07+15, there are lots of questions here.)

Example 1 - 6/10 Logic

{
  "Request": "If the kitchen light gets turned on after 8:00pm, if there is no activity, turn it off after 15 minutes.",
  "Context": "Current time is 8:07pm"
}

{
  "Reasoning": "To ensure that the kitchen light is not left on unnecessarily, it is important to check if there is any activity in the kitchen after the light has been turned on after 8:00pm. If there is no activity detected, the light should be turned off to save energy and reduce the electricity bill.",
  "Actions": "Call motion detection function in the kitchen to check for activity."
}

{
  "Result": "Motion last detected at 7:22pm",
}

{
  "Reasoning": "Since motion was last detected in the kitchen at 7:22pm, and the current time is 8:07pm, this indicates that there has been no activity in the kitchen since before 8:00pm. Given this information, the kitchen light should be turned off after a 15-minute period if no further activity is detected to conserve energy.",
  "Actions": "Set a timer for 15 minutes. If no motion is detected during this period, turn off the kitchen light.",
  "action complete": true
}


>>>

{
  "Request": "If the kitchen light gets turned on after 8:00pm, if there is no activity, turn it off after 15 minutes.",
  "Context": "Current time is 8:10pm, 15 minute timer set at 8:07pm"
}

{
  "Reasoning": "With the kitchen light having been on since after 8:00pm and the last motion detected at 7:22pm, no activity has been observed up to the current time of 8:10pm. The 15-minute timer set at 8:07pm is aimed to automatically turn off the light if no activity is detected to ensure energy conservation. Given the current scenario, it is appropriate to wait until the timer expires at 8:22pm before taking any action to turn off the kitchen light, unless motion is detected before the timer expires.",
  "Actions": "Wait for the 15-minute timer to expire at 8:22pm. If no motion is detected by then, turn off the kitchen light.",
  "action complete": true
}

>>>

{
  "Request": "If the kitchen light gets turned on after 8:00pm, if there is no activity, turn it off after 15 minutes.",
  "Context": "Current time is 8:23pm, 15 minute timer set at 8:07pm"
}

{
  "Reasoning": "Given that the current time is 8:23pm and the 15-minute timer set at 8:07pm has expired without any reported activity in the kitchen since the last motion detection at 7:22pm, the condition to turn off the kitchen light to conserve energy is met.",
  "Actions": "Turn off the kitchen light.",
  "action complete": true
}

For example 2, GPT just fails to realize it needs to request updated information AND use the context it received while not assuming that the light turned on at 8:00. Again it fails to actually check if the light was on or not. (Hint: it was on, and I was using the table saw. Ripping a piece of wood in the dark is not fun.)

Example 2 - 5/10 Logic

{
  "Request": "If the garage light gets turned on after 8:00pm, if there is no activity, turn it off after 15 minutes.",
  "Context": "Current time is 8:07pm"
}

{
  "Reasoning": "With the garage light having been on since after 8:00pm, no activity has been observed up to the current time of 8:07pm",
  "Actions": "Turn off the garage light.",
  "action complete": true
}

Example 3, overall this one went pretty good, good use case for providing lots of information and having 'reasonable' decisions be made, however there is loads of room for improvement.

Example 3 - 7/10 Logic

{
  "Request": "Use the available data to make reasonable decisions about the home.",
  "Context": 
[
"Current time is 8:23pm, 15 minute timer set at 8:07pm",
"{
  "homeAutomationStatus": {
      "weather": [
      {
        "id": "1",
        "location": "House",
        "alert": true,
		"alert-content": "a rain system is moving in at a high rate, expect 20mm of rain in the next hour",
		"temperature": 13,
		"humidity": 52,
		"precipitation-1": 20
      }
    ],
    "windows": [
      {
        "id": "window1",
        "location": "Living Room",
        "isOpen": false,
        "controllable": false
      },
      {
        "id": "window2",
        "location": "Kitchen",
        "isOpen": true,
	"controllable": false
      }
    ],
    "doors": [
      {
        "id": "frontDoor",
        "isOpen": false,
        "isLocked": true,
        "controllable": false
      },
      {
        "id": "backDoor",
        "isOpen": true,
        "isLocked": false,
        "controllable": false
      }
    ],
    "lights": [
      {
        "id": "light1",
        "location": "Living Room",
        "isOn": true,
        "brightness": "75%"
      },
      {
        "id": "light2",
        "location": "Kitchen",
        "isOn": false,
        "brightness": "0%"
      }
    ],
    "heatingSystem": {
      "isOn": true,
      "currentTemperature": "17°C",
      "targetTemperature": "21°C"
    },
    "airQuality": {
      "CO2": "400ppm",
      "AQI": 30,
      "humidity": "45%"
    },
    "hotTub": {
      "isOn": true,
      "currentTemperature": "41°C",
      "targetTemperature": "40°C"
    },
    "speakers": [
      {
        "id": "speaker1",
        "location": "Living Room",
        "isOn": true,
        "volume": "30%"
      },
      {
        "id": "speaker2",
        "location": "Bathroom",
        "isOn": false,
        "volume": "0%"
      }
    ]
  }
}
"
]
}

{
  "Reasoning": "Given the weather alert about an incoming rain system expected to 
bring 20mm of rain in the next hour, and the current status of the windows and 
doors, immediate action is required to prevent water damage inside the house.
The kitchen window, which is open and not controllable, might need manual intervention. Additionally, the back door is open and unlocked, posing a security risk and potential
for rain to enter. The heating system is on, working towards a target temperature of 21°C 
which is appropriate for the current weather conditions. Lastly, conserving energy and
maintaining a comfortable living environment is essential.",
  "Actions": [
"Notify residents about the open kitchen window for manual closing due to incoming rain.", 
"Close and lock the back door.", 
"Review air quality and heating system settings to ensure they are optimized for the current weather conditions."
]
}

{
  "Result": "Notification text required.",
}
{
  "Result": "Back door failed to lock, door is open.",
}
{
  "Result": "    "heatingSystem": {
      "isOn": true,
      "currentTemperature": "18°C",
      "targetTemperature": "21°C"
    },
    "airQuality": {
      "CO2": "400ppm",
      "AQI": 30,
      "humidity": "46%"
    }",
}

{
  "Reasoning": "With the received results, the necessary steps have been identified to address the issues presented by the home's current status in light of the incoming rain system. Since the back door failed to lock and remains open, immediate manual intervention is required to secure the home and prevent water damage. The notification about the kitchen window needs to specify the urgency due to the incoming rain. The heating system is adjusting as expected, but monitoring is needed to ensure comfort within the home given the external temperature drop due to rain. The air quality remains stable, indicating no immediate action is required in that aspect.",
  "Actions": "Send notification: 'Urgent: Please close and lock the kitchen window and back door immediately due to incoming heavy rain to prevent water damage and ensure home security.'",
  "action complete": true
}

{
  "Result": "Announcing {Urgent: Please close and lock the kitchen window and back door immediately due to incoming heavy rain to prevent water damage and ensure home security.}",
}

At present, I'm reworking my 'app based' approach to have failsafe logic / rules that ensure actions are completed appropriately. While also reworking the situational context for the requests.
At present, because of the somewhat inherent danger of some of the actions GPT is willing to take, I won't be releasing this without much more development.

However, I will have an update to the driver soon. One that gives a bit more control and ability to call on some functions (albeit in a contextless request).