Usage Data Pipeline

Hello, all. Is there a way to export data for storage on usage within the home? In other words, can I somehow stream or export usage data (i.e. lights turned on and off, motion events, etc.) to something like a datalake to be used for ML or reporting later? I haven't seen anything like that, so wanted to ask. While I realize there are other more mainstream features needed, but I think this would be cool to be able to get insights in to how the home is being lived in, how the smarthome is used by the people living there, and using predictive analytics, maybe even make the smarthome smarter going forward. I'm not even sure how I would use all the data yet, but would like to start gathering it for future use. Thanks.

If you are willing to use node red, see this post:

With this method I was able to generate this so far, and plan to add quite a bit more with time.

I use node red to connect to 2 web sockets. One for events (open, close, motion, etc) and the other to collect logs so I can maintain weeks of logs very easily.
My event socket exports to Influxdb database and I use Grafana to create graphs, etc. Works really well and does not bog down the hub like the ported Influxdb app did.

My log socket exports to a delimited text file that I Import into excel where I can sort and manipulate it easliy.


Bump! I'm also interested in device data experts for use in Tableau.