Man, I just picked up a Sony Android TV with 5.1.1 on it and I have to say – casting to this thing kind of sucks compared to an actual Chromecast. It's so flaky – sometimes it'll connect and then drop straight away, or it'll connect but stop playing music, or it'll just disappear off the network entirely as a casting device. What gives?
The other thing I'm annoyed about is that Android TV has no way to set your own custom backdrop images like a Chromecast does. I expected it to show up in the "Home" app (previously Google Cast) like all other cast devices so I can configure the backdrop photos. Really disappointed.
I had exactly the same experience with my Sony but it's been perfect since I updated the TV's cast app. Go to the Play Store on your TV, go to My Apps and update Google Cast Receiver. I have no idea why this app doesn't auto update but updating it manually makes a world of difference.