Can't connect xbox controller, (keeps disconnecting and connecting)

I have the common problem where the controller keeps doing this:

[CHG] Device 5C:BA:37:81:C3:DC Connected: yes
[CHG] Device 5C:BA:37:81:C3:DC Connected: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: yes
[CHG] Device 5C:BA:37:81:C3:DC Connected: no

I have tried everything for many months, so now i'm finally asking for some help :(

disable_ertm is set to Y (I've tried N too, same result). I'm on 5.5.7 kernel but i've tried many kernels and some kernel modules for my rtl8822be chip.

One big problem is that I can't re-pair the device, whenever i type remove, the controller is automatically paired again!

[bluetooth]# remove 5C:BA:37:81:C3:DC
[DEL] Device 5C:BA:37:81:C3:DC Xbox Wireless Controller
Device has been removed
[NEW] Device 5C:BA:37:81:C3:DC Xbox Wireless Controller
[CHG] Device 5C:BA:37:81:C3:DC Modalias: usb:v045Ep02E0d0903
[CHG] Device 5C:BA:37:81:C3:DC UUIDs: 00001124-0000-1000-8000-00805f9b34fb
[CHG] Device 5C:BA:37:81:C3:DC UUIDs: 00001200-0000-1000-8000-00805f9b34fb
[CHG] Device 5C:BA:37:81:C3:DC ServicesResolved: yes
[CHG] Device 5C:BA:37:81:C3:DC Paired: yes
[CHG] Device 5C:BA:37:81:C3:DC ServicesResolved: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: yes
[CHG] Device 5C:BA:37:81:C3:DC Connected: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: yes
[CHG] Device 5C:BA:37:81:C3:DC Connected: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: yes
[CHG] Device 5C:BA:37:81:C3:DC Connected: no
[CHG] Device 5C:BA:37:81:C3:DC Connected: yes

If i type pair, i get this:

[bluetooth]# pair 5C:BA:37:81:C3:DC
Attempting to pair with 5C:BA:37:81:C3:DC
Failed to pair: org.bluez.Error.AlreadyExists

I don't know what else to do! I have 2 controllers that works fine on Windows but never worked on Linux. So frustrating so I hope someone could give me some direction.

Thanks!