[RELEASE] Google Assistant Relay 3.3 w/ CATT Integration

There have been some big changes to Google Assistant Relay (GAR) recently that I'd like to share with everyone. The biggest changes are changes to the install process, the addition of a server dashboard including a sandbox area and finally integration with CATT to stream media and websites to your Google displays, speakers and Chromecast devices.

This integration with CATT is HTTP call based, just like GAR is. So no persistent Telnet connection to your server is required from Hubitat. A connection is only established to send the command to GAR and then closed.

Let's start with how to install/upgrade GAR. There is all-new documentation to help you get started. It can be found here. Greg has done a lot of work on the updated documentation, so I'm not going to re-hash all of that here except to mention this: v3.3 and higher of GAR will require that you have Python 3.7 or higher installed on your NodeJS server. CATT is python based and in order for it to run you have to have Python. Even if you don't use any of the functions of CATT, you have to have Python installed. There is no longer a branch of GAR that doesn't include CATT. So, if you don't want to install Python, I would not upgrade your GAR.

If you are running GAR on docker, I do not know of a docker image for the new version yet that runs on a Raspberry Pi. Hopefully someone will come up with something but nothing exists at this point to my knowledge.

To get started with the Hubitat integration, follow the instructions in the how-to up through and including configuration. If you have used a previous version of GAR, you do not need to create a new Google Console project. Your JSON file can be used for this new version of GAR as well.

Next, test your setup using the Sandbox on the new GAR Dashboard. This new feature allows you to send broadcasts to your speakers without the involvement of Hubitat, thus eliminating one point of failure. If you are unable to get broadcasts to work from the Sandbox, do not proceed with the installation in Hubitat! If you are unable to get broadcasts from the dashboard, you won't be able to get them from Hubitat either. So, if you are having issues, stop there and get those resolved before proceeding and save yourself a lot of headaches. Once you have GAR up and running and are able to issue broadcasts from the Sandbox to your speakers, you are ready to set up Hubitat to talk to GAR. You can locate the latest drivers that I have developed here:

You will see both a parent and child device driver file. Install both to your system, installing the parent driver first. Once you have installed the drivers, create a parent device the same way you would create any other virtual device using the Google Assistant Relay driver. Once created, go into your device and specify all of the parameters necessary in the driver (IP, Port, Username) (Tag A)

Functionality of the Driver

First off, there has been no change to the existing features of the GAR driver originally released by @ogiewon. The custom commands still work with [CC] and custom commands with converse true still use [CCC]. You can still issue broadcasts the same as you did in V1 and V2 via either the Speak command or the Device Notification command. Those are all the same.

The new features are all related to Casting Websites and Media to your Google speakers and displays. That can be accomplished in 3 different ways, each of them becoming simpler from a daily use perspective but more complicated from a setup perspective.

Option 1 - Directly from the parent device

There is a custom command in the parent GAR device called Cast Content. There are 3 parameters you can send along with this command. They are: device name to cast content to, type of content to cast and source of that content. (Tag B)

The device name must either be the IP address of your Google device or the proper device name, properly cased to match whatever name you have assigned to the device in the Google Home app. The content type must be one of 3 options, website (for a website), remote(for remote media or non-standard local media) or local (for locale media on a Windows machine). The last option, local, I will not be going into since this does not work when you run GAR on a Linux based system like a Raspberry Pi. However, the remote option does work with local media. You just have to do one special thing in the source parameter.

Which brings us to source. Source is either the full URL of the website you want to cast to the display. Or the URL or filepath of the media you want to cast. The URL must be a direct download (playable) link (no Dropbox or Google Drive links to splash pages will work). You can also use the URL for any local dashboard or the URL of any accessible HTTP camera stream (RTSP will not work). You can also use the full file path for any local media on your nodeJS server . Let me say that again as to be clear...the media must be on your NODEJS server or accessible via HTTP. You cannot use an FTP URL or Samba share or any other means to share media to the server. Only an HTTP url or a local (to the server) filepath will work. Currently, the option "local" only works for Windows based Node JS servers. In that case, the media must be in the "media folder" inside the GAR folder on your server and you only need to specify the filename. To play media located in other folders on your server or if you are using a Linux based server, like a Raspberry Pi, select the "remote" option and just enter the full file path to the media. If you are unclear on this, there are more details in the GAR how-to linked to above.

2 of the parameters are required for the Cast Content command, Type and Source. A default device can be specified in the driver and if not supplied in the command, that device name/IP will be used. (Tag C) The other two must be specified each time you send the command from the parent device.

Option 2 – 1 Child Device for 1 Google Device

The second and simpler way to cast media to your speaker, speaker group or display is with a child device. In the parent device there is a command to create a child device after specifying the name of the new child device. (Tag D) Once created, you can go into the child device to specify a few things.

In the child device there is an setting for the cast device name. (Tad E) Fill this in with your speaker, speaker group or display name or IP address. You can then use the commands Cast Website and Cast Media with the source parameter being the website or media URL/filepath that you want to play/display. (Tag F) Once you issue those commands, the switch attribute will switch “on”. We'll get to switching it off a bit later.

Option 3 – 1 Child Device for 1 “Thing”

The final option, and by far the simplest, is to create one child device for each “thing” you want to display/play. You create a child device the same way you do for option 2 but then you also specify the default type and default source in the child device settings. (Tag G) If the child device is turned on via the normal switch command, the default content will be cast to you display/speaker. You can create as many child devices as you like. I have one for each “thing” I typically cast. So, one each for 3 dashboards and 2 each for 2 cameras streams. That way, I just have to turn on the appropriate child device to get that content shown on by home hub display.

Whenever you begin casting to a Google Device from a child device either with the custom command or the switch, the switch attribute will turn on. Also available in the child device are 3 additional features which make it easier to control the stream.

The first is an option called Max On. (Tag H) This option lets your specify, in seconds, the maximum time the the switch should remain on. The switch attribute on the device will automatically turn off at the end of this timeout. You can also turn the switch off manually. If you have max on defined but turn the switch off early, the scheduled shut off will be canceled.

When that switch is turned off, via the option Enable Stop (Tag I), you can have a stop command sent to your Google device. For displays, this will remove the website from the screen or the cast icon for media. This is also useful for speakers if you are using your Google Speakers as alarm or beep device with an alarm sound. Stopping will stop the media from playing. However, you should note, there is a good 2-4 second delay from the time the stop is requested by Hubitat to the time the display/speaker stops playing. This is the time it takes GAR and Catt and Google Assistant to process the request. It is as fast as it can get at the moment, unfortunately.

The stop command is also available in the parent and child devices. In the parent device you must specify the Google Device name otherwise the default device specified will be use. In the child device, the Google Device specified in the properties will be stopped. There is also no automatic stopping if you use the parent device. Since that device can cast to multiple Google devices, it because too hard to try which one had to get stopped when so you have to do it by hand.

Side Note: I have decided to only use stop as a custom command and not use the full Music Player capability here. There are too many other commands and attributes required to make that effective. However, that means that the stop command is only accessible in Rule Machine as a custom action, not via "Control Music Player".

The final option available is Enable Mute (Tag J). This allows you to mute your Google device before casting to it. This option is not so useful for any media that involves sound (or useful at all for speakers) but great for content that doesn't include sound. Are you as annoyed as I am by the “Cast Tone” that happens when you cast a dashboard to your Google Home Hub? The bong that is played before the web page is displayed? Well, Enable Mute eliminates that sound from playing before casting the media to your display. If enabled, your display will be unmuted when the switch is turned off, either automatically or manually. This is really useful for both dashboards and camera streams. One thing to note here, muting does not affect Broadcasts issued through GAR. Those are only muted if you turn on Do Not Disturb on the speaker/display. So, this will not interfere with any broadcasts.

The mute and unmute commands are also available in both the parent device and the child device. Similar to Stop, in the parent device you can specify the Google Device to be muted/unmuted and if not specified the default device will be used. In the child device, only the device specified in the device properties will be commanded to mute/unmute.

The easiest way to set up GAR with this is to use option 3 with the options Enable Stop and for dashboards Enable mute, enabled. That way, in other automations or manually, you only have to turn on the child device to get your content to display on your display or media to play on your speaker. This is what I have done.

It should also be noted, that there is no keyboard on your Google Nest Hub. So, you cannot have any type of pin associated with your dashboard. They must be able to be displayed automatically. You can, however, still have the pin enabled for HSM or Mode control as both of those have an on-screen keyboard to enter the pin. When trying to display a camera, if the stream requires basic authentication, you can enter that into the URL for the stream. For example, if your camera is displayed by going to the URL http://192.168.1.200:8081/feed/1 but you have to enter a usermane and passwords, you can encode them into the url like this:

http://username:password@192.168.1.200/feed/1

This works for all sites that use a basic authentication model. If your camera requires something more complicated for authentication, it may not be able to be displayed. Also, camera streams only work if in HTTP format. RTSP camera streams cannot be displayed.

If you want to play media to more than one speaker, I recommend creating a speaker group rather than trying to cast to multiple speakers at the same time. If you create a group, Google will handle syncing the playback. Otherwise, there is no way to insure that they are all synced. This means, however, is the sound file is short, it may not play on all speakers.

Feel free to take an modify the driver if you wish. I only ask that if you do, you make sure to include credit to Greg as he is the one who should really be thanked for all this. Without him, there wouldn't be GAR. I will not be accepting any donations. If you really appreciate the drivers I've built and want to contribute to someone, I ask that you send those donations to Greg. Details are in the How-To linked to above. You can, however, mention to him that you're a Hubitat user and using my drivers. That I would appreciate. :blush: If you wish to thank him directly, he is active on the ST forum under the username ghesp.

If you think you have found a bug in the driver, the first thing you should do is check that the command you are trying is possible via the Sandbox on the GAR dashboard. If you can't do it from the Sandbox, that means that GAR can't do it. There are many, MANY limitations that Google has placed on the Google Assistant SDK (the backbone of GAR) that are completely out of anyone's control. There is a list of known limitations on the Github page for the drivers. Please review it before submitting a bug report.

Post any questions you have either on this thread or in Github.

I worked a long time on these to make sure they worked right, so please be kind. And enjoy!!

Driver Screenshots

Features In Development

Features In the Works:

  • Play media at a specific volume
  • TTS Annoncements to Individual Speakers
4 Likes

Saved

Hello @Ryan780:
I seem to be having an issue in installing the RPI portion of the package.

  1. I've got all the requirements installed.
  2. When I try and run a sandbox command all I get is a brief, small window with a X in it.

Any ideas?

First of all, the "user" that you are entering in the first box, does that match the name that you entered when you uploaded your JSON file in this window?

image

Typically, folks don't user their full email address for their username, they just use their first name. I don't know if a username with an @ sing in it will actually work correctly. Also, you can't issue both a preset and a custom command at the same time. You can do one or the other but not both. There is a full description of how to use different commands here.

  1. Yes, my email address is what I used to upload my JSON file.
  2. I checked with and without a custom. Same error message.
  3. In order to change it from my email address, I'm going to have to re-create the google project?
    Is that correct?

No, you would just have to upload your JSON file again using just your name.

A few low-hanging issues that are common problems.

  1. Do you have IPv6 enabled on your router? If so, disable it and try again. This is a known issue with GAR that will prevent it from working.
  2. Are you Google devices on the same network where you are running GAR?
  3. Do your speakers and your server have static IP addresses?
  4. Are you able to ping the speakers from your server?
  5. Do you have any network isolation on in your network?

That will cover about 90% of the issues that folks have getting GAR up and running.

But I just saw something that might be the cause of the problem. What you want to do is remove what you have under custom and toggle both broadcast and converse on and then off again. Your screen should look like this:

Then you should be able to get the broadcast. I too got the red x without toggling both broadcast and converse on and off again. Otherwise, try this:

image

That should also work correctly.

I had this same issue when I installed it and after recreating the user a few times and rebooting my pi as well it finally decided to work.

Also check that once it does work that you can still use it after a few commands as I had issues with it again throwing an error twice before it worked for good.

Thanks for your work on this.
I'm not sure if it's a bug or by design but it seems you need to stop casting before you can cast another website/media to a device.

Maybe call out the NFS will work as I'm sure people who are using a RPi and haven't ever used linux (like me) might not understand that you can share media from a NAS/Network if you set it up using NFS.

That is a limitation of CATT. It only casts one thing at a time.

No, it must be the local filepath or the HTTP playable link. Those are the only things that work. A network share is not accessible from the speaker/display. CATT just tells the speaker where to get the media. The Google Cast functions doesn't support NFS shares.

@Ryan780 Hey there Ryan. I just installed this and am wondering if something happened to the device drivers.

1 Like

I appreciate the detailed instructions. I can't seem to find the Hubitat drivers for V3 of the assistant relay. Or am I confused and should be using the drivers for V2?

1 Like

I'm getting following during build of 3.3 on raspberry pi:

node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp WARN Tried to download(404): https://node-precompiled-binaries.grpc.io/grpc/v1.24.2/node-v83-linux-arm-glibc.tar.gz
node-pre-gyp WARN Pre-built binaries not found for grpc@1.24.2 and node@14.1.0 (node-v83 ABI, glibc) (falling back to source compile with node-gyp)

If I manually go to the web page I get this:

NoSuchKey

The specified key does not exist.

No such object: node-precompiled-binaries.grpc.io/grpc/v1.24.2/node-v83-linux-arm-glibc.tar.gz

I'm wondering the same... Hoping that you still have these available, would appreciate being able to capitalize on the work you've done here!

Ryan has a indefinite ban here so he pulled all his code.