I have not had an issue with it not speaking, but you can assume that it is a symptom of being in beta. Recently I setup cast-web-api. It was fairly simple, especially since I already had setup the client ID and secret from Google Assistant Relay. It is working really well and I really like that it now allows you to cast messages in the Google Assistant voice as an option.
If you're not opposed to running an extra Raspberry Pi or old laptop as a node.js server, then cast-web is a great option that takes the additional load off of your hub. If you have not setup a node.js server, this is a good reason to start. I'm not running four different instances on mine for Google Assistant Relay (super useful), Insteon-server (excellent in-wall products), Homebridge MakerAPI (working fantastic without loading down the hub), and Cast-Web-API (Native Google Assistant voices on individual devices, individual volume control, and broadcast capability too).