Uninstall any antivirus, antivirus on Android and iOS are scams. Google Play Store integrates a virus scanner out of the box .

Which apps did you install that needed so many permissions? Most of the apps I have don't require any permission at all, except for the microphone for the voice search.

Also, you can fine tune the permissions on the Android apps, so if you think an app doesn't need a specific permission you can just prevent it from accessing it.

More info: https://www.quora.com/Do-Android-phones-need-antivirus-or-internet-security-apps

I noticed that most of the apps are requiring way too much intrusive permissions that the app won't ever need

Do you have some examples? They only thing I've noticed is some apps that are made for both ATV and phones will ask for the same permissions as on your phone, but the ATV doesn't normally have what they need anyways. There's no camera, no contacts, no real location, etc.

I'm trying to stay safe by using only apps from Google Play Store but I still kinda don't want to miss the fun of it

If security is highly important to you, you should never sideload apps, obviously. The only thing I sideload is RetroX. Everything else I get from the Play Store either directly on my Shield, or by pulling the official APK from my phone and loading via AirDroid.

I installed Eset Antivirus but not really sure if it's effective or not

Please don't install AV apps on your streaming device. They are a placebo do to the nature of how Android and the Play Store function.

I only use official apps like Netflix, Hulu, YouTube, etc. so I have zero concerns about security. I don't see a need for antivirus on any Android device, whether phone or tablet or Android TV. Unless you're installing a bunch of 3rd party APKs from random sites? Then yeah, you're just asking for trouble. You cannot trust those apps at all anyway.

Remove antivirus.

Google will eventually catch any play store malware, thus avoid brand new Google store apps or apps with low amount of users.

If you sideload apks, only use reputable apk sites like apkpure and apkmirror.

SMH. You are so worried about security that you made your OS less secure by installing antivirus. Guh.

Antivirus programs are a scram to grab your information. It's 2018. Android doesn't need them, IOS doesn't need them and even Windows 10 does not need them. Your OS already takes care of you.