I think Google is moving the remote control functionality for TVs to the updated Google TV app. I also had problems with Google Home remote and switched to Google TV remote and it works more reliably.

So, if we want to cast screen to TV, we have to use Google Home app and if we want to remote control TV, we have to use Google TV app. What a mess!

You need to update "Android TV remote service" on your TV

Edit : Google Tv Remotes works fine and its an improvement