I'm looking for advice on integrating a deep learning AI camera into home automation setups for Hubitat. The camera has the capability to divide its field of view into multiple zones—up to 20 different areas. In each zone, it can detect if there's a person present, whether they're moving or stationary, also cars, trucks or bikes. This will work inside or outside.
My goals are:
Scene Integration: I want to set up scenes that react based on the presence or movement of a person in specific zones. For example, turning on lights when someone moves around a space or adjusting security settings based on proximity. In large inside areas it can detect which area the person is in, so as they move around the space it can adjust the automation. These cameras can track multiple targets.
Outside: it can detect a car while it moves and when it stops, it can also tell if a person exits the vehicle. You can imagine what automation's can be done from this information. 1000x what a simple motion sensor can accomplish.
Home Automation Control: I'd like the camera to relay information to my home automation system so it can control lights, security features, and other connected devices accordingly.
I'm seeking guidance on the following:
Integration Methods: What are the best practices for integrating such an AI camera with a home automation system? Are there specific protocols or standards (like MQTT, REST APIs, http, or others) that facilitate this kind of communication?
Compatible Platforms: Has anyone integrated a similar camera with platforms like Hubitat, Home Assistant, or others? Which platforms offer the most seamless integration for this use case?
Middleware or Hubs: Do I need additional middleware or a specific hub to bridge the camera's data with my home automation system?
Automation Setup: How can I configure automations or routines that respond to the camera's zone-based detections? Any tips on setting up these conditions effectively?
Challenges and Solutions: What potential obstacles should I be aware of during this integration? Any advice on troubleshooting common issues?
I'd greatly appreciate any insights, experiences, or resources you can share to help me achieve this integration.
I have been digging further into this, see if this makes sense for the integration:
Camera's HTTP Integration Capabilities
HTTP Notification Features:
The camera can send HTTP notifications to a specified server when events are triggered. Will this allow integration with Hubitat to perform actions based on camera detections.
Tokens Available for HTTP Notifications
Tokens are placeholders that are replaced with actual event data when the notification is sent.
{{name}} – Event name.
{{id}} – Unique event ID.
{{type}} – Event type (e.g., rule that triggered the event).
{{status}} – Event status.
{{iso8601}} – Event time in ISO 8601 format.
{{time}} – Event time in Unix timestamp format.
{{ip}} – IP address of the camera.
{{host}} – Hostname of the camera.
{{datetime}} – Event time in formatted date and time.
{{zoneid}} – ID of the zone where the event occurred.
{{ruleid}} – ID of the rule that was triggered.
{{bb}} – Bounding box coordinates of the detected object.
{{objectclass}} – Classification of the object (e.g., person, vehicle).
Select the devices you want to control or monitor.
Note: Obtain the Local Endpoint URL and Access Token.
Create Hubitat Rules:
Use Rule Machine to create rules triggered by HTTP requests.
Trigger Conditions:
Set up to respond to specific HTTP requests from the camera.
Actions:
Define actions like turning on lights, sending notifications, etc.
Parsing the Incoming Data:
Extract parameters from the HTTP request (e.g., zone, event, object).
Use variables in Rule Machine to make decisions based on these parameters.
Questions for Hubitat Developers and Community
Best Practices for HTTP Integration:
What is the recommended method for parsing HTTP request parameters in Hubitat?
Are there built-in features or community apps that simplify this process?
Custom Drivers or Apps:
Do I need to create a custom driver or app to handle the HTTP notifications and extract token data?
Are there existing drivers or examples I can reference?
Security Considerations:
What measures should I take to secure the Maker API endpoint?
Is it possible to limit access to the endpoint or use authentication methods?
Performance Impact:
Will frequent HTTP requests from the camera affect Hubitat's performance?
Are there strategies to manage high-frequency event data?
Alternative Integration Methods:
Is there support for other protocols like TCP, ONVIF, or SNMP with Hubitat?
Would using MQTT or another intermediary service be beneficial?
Thank You:
I appreciate any guidance or examples you can provide to help me achieve this integration. Your expertise is invaluable in enabling advanced home automation features.
Don't know if this helps, but I read this article a while ago about someone you used ML to deter foxes and badgers from his garden. There might be some tips in this that are useful.
You probably want to review the Frigate NVR solution. Running this takes some horsepower and the HA folks use bigger rigs to get acceptable performance. Good discussion.
This seems like a reasonable way to integrate with Hubitat. You could make use of the Maker API or rule machine rules triggered by an http endpoint, as you’ve noted.
It is pre-loaded with remote view (NVR storage on Camera). Also the AI app is onboard as well. It is a very self contained commercial camera. Kind of unique
I realize no one here has possibly integrated this AI camera in the habitat, however I was curious based on the information I provided here what you guys feel is a probability that the integration would work
I'm curious what this camera costs... I can't find a price on-line, but for other models by the same company, these would seem to be extremely expensive for the feature set provided.
Cost at a retail level would be around $2000.00 at least that is what I would install it for.
You have to be a licensed low voltage security dealer to purchase, There are other models that still have AI on edge analytics on camera that require an NVR to record that are far less money.
I work with high end clients which this does fit their budget
In fact the project I am going to implement this on has grown from one camera to 10 so I will choose one of those or a combo of them
I am planning on looking at all my CCTV suppliers with AI and dig in to see if they have outputs like http to integrate, most don't its just on their NVR
I'll admit I haven't read this thread in detail, but that sounds like one hell of a camera. The inbuilt http and tokens sound like all you'd need to use say, webcore to create a bit of magic and everything you need.
/ jealousy
Good luck and it would be amazing to see the end result. Wow.
If you (or someone you know) is interested in writing a driver for this (and perhaps other) camera(s), then I think you would have an even better outcome, with the potential (albeit low) for the camera to become a built-in integration, like Kasa and Bond have become.
It feels like the outputs from the camera could be relatively generic in that they represent people or other objects moving through defined spaces, not something completely uncommon in other sensors like an Aqara FP2. An app/driver that simply accepts inputs around these more general concepts could potentially be used across more than just this camera....
I would hope for the msrp the built-in AI for object detection is waaaaaaay more sophisticated than the products empiretech sells, which I believe are gray market Dahua cameras (maybe Hikvision too).
Personally, I use a computer for NVR and AI object detection capabilities with cameras in the price range that are similar to what you linked to.
I don’t really see the appeal of putting those features on each individual camera for a home security setup, particularly given the exorbitant price point to get those features on the camera.
As a CCTV installer I thought similar as most Hikvision and Dahua cameras have a lot of the functionality (and have an API that can be utilized). The camera in question isn’t available in the UK, but there are some features in the specs of interest that are not available in most IP cameras. On board DVR functionality- sure you can pop a 512GB SD in an IP camera for on board recording but that’s not the same as a camera that hosts its own recording app (most IP cameras are more feature rich when using the vendors NVR). 40 Zone detection - Hikvision have just 4 per VCA (motion, line crossing, intrusion) and the deep learning AI.
Axis Communications products have a long (but always expensive) history of working towards "smarter standalone camera platforms" that allow communication/action/and polling outside their universe.
Their client focus has long been very high end security but I've actually been happily surprised by what, in the way of recent AI development, they are bringing down into their more affordable platforms.
There's a number of vendors that SAY their inexpensive cameras can do this, but the case here is that Axis is FIRST refining this technology up on their very expensive platforms (to the point that it satisfies organizations & agencies for which security is paramount across the globe) then rolling it down the product line as the internal computational capabilities allow.
So after just setting up some HE to AXIS VAPIX (HTTP) stuff with a new Axis camera... I got to reading about all the ONVIF Profiles their cameras support and that got me wondering and to this thread today (after searching on ONVIF in the Forums).
Now I'm pondering what, if anything, might be worth the effort in ONVIF Profile(s) certification to open unique doors for Hubitat and facilitate a lot of what's being talked about in respect to integrating increasingly smart cameras within the Smart Homes that HE resides. @bravenel
