So either I'm crazy or the audio button on the FireTV remote is working and it didn't before. It was previously done by holding the play button and was hit or miss, now it works every time.
Now if only I could get volume buttons to control the TV's volume using IR.
Mic buttons activates voice search but isn't listening