I got off a long troubleshooting chat with Google on the subject issue. To save others the time, here's the relevant info:
Google Response: "Our engineering team is aware of the issue where customers who have built in Chromecast for Vizio, Sony and Nvidia Shield products are having problems using voice commands to cast on the TVs. The product and engineering team are already aware of it and are coordinating this with the said partners. As of now, we are still waiting for their investigation and review to find a stable fix."
To help them with their investigation, for anyone with the issue they wanted me to do the following:
"We would need your help with investigating this further by filing a feedback from your Google Home app. Just open the app and at the top right, tap your account. Tap "Help & Feedback" and then "Submit feedback report (under the Google Home section) with the subject: "CC T3 Unable to recognize voice command". Also list the firmware version of your TV and built in Chromecast"
Other relevant info: I tried all the solutions in this thread, and it didn't work for me. Feel free to try in vain like I did if you like. https://piunikaweb.com/2019/11/20/nvidia-shield-google-home-integration-is-broken-for-some-but-theres-a-way-out/
Nvidia support didn't really help me at all. They did confirm a separate issue where the Shield cannot display Google Photos in its screensaver using the "Photo Frames" from Google Photos. They said they were aware, and can't comment. They wouldn't provide an ETA on this. For the voice command issue, they told me to contact Google, but they weren't very helpful and didn't seem to take it very seriously.
Had this same issue shortly after getting my Philips Android TV. Spoke at length to Google and Philips, both of them eventually giving up on contacting me and eventually I gave up chasing them up.
One day recently the voice commands started working again. No idea what the change was.