Same issue here when watching live streams. Very annoying!
Yes that's happening to me too but only when watching Live streams. It doesn't happen on resolutions below 480p.
Just use SmartTubeNext. Sideload it.
It doesn't have any ads and has tons of options. And there is like 3x more choices for resolution. It has PiP mode support, seeking is better because it has so-called carousel preview.
Great app while Google's YT ATV app is ****.
Whew! I thought that it was just me ...