How to actually connect an already paired Xbox controller to Mi Box 3?

I had no problems pairing the controller over Bluetooth and could use it just fine. But I have no idea how to connect the controller again after turning the TV on the next time - the controller is there among the paired devices, but won't connect automatically either in normal searching mode or pairing mode. If I select it in the menu it just offers to change the name or unpair it, I don't think unpairing and pairing again is the solution here. Should be simple, what am I missing?