Interestingly I found the solution, the problem was:
Full insertion had some problems with communication, specifically usb descriptors weren’t able to get read.
Partially pulling it out solved the problem! Got the idea by looking at logs doing connection again and again. After partial pull and let the mouse communicate properly, pushing it all the way back later on has no effect and it still work perfectly.
2022-03-20T17:13:03.27343 kern.info: [52135.347098] usb 3-4: new full-speed USB device number 47 using xhci_hcd
2022-03-20T17:13:03.67153 kern.info: [52135.745565] usb 3-4: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=12.11
2022-03-20T17:13:03.67157 kern.info: [52135.745571] usb 3-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
2022-03-20T17:13:03.67157 kern.info: [52135.745573] usb 3-4: Product: USB Receiver
2022-03-20T17:13:03.67158 kern.info: [52135.745574] usb 3-4: Manufacturer: Logitech
2022-03-20T17:13:03.67739 kern.info: [52135.751996] logitech-djreceiver 0003:046D:C52B.00D2: hiddev0,hidraw2: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:14.0-4/input2
2022-03-20T17:13:03.80654 kern.info: [52135.880987] logitech-hidpp-device 0003:046D:4055.00D3: HID++ 4.5 device connected.
2022-03-20T17:13:03.94940 kern.info: [52136.023140] input: Logitech Wireless Mouse as /devices/pci0000:00/0000:00:14.0/usb3/3-4/3-4:1.2/0003:046D:C52B.00D2/0003:046D:4055.00D3/input/input89
2022-03-20T17:13:03.94943 kern.info: [52136.023463] logitech-hidpp-device 0003:046D:4055.00D3: input,hidraw3: USB HID v1.11 Mouse [Logitech Wireless Mouse] on usb-0000:00:14.0-4/input2:1
2022-03-20T17:13:07.40352 kern.info: [52139.477419] usb 3-4: reset full-speed USB device number 47 using xhci_hcd
2022-03-20T17:13:07.78355 kern.err: [52139.857009] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:08.00559 kern.err: [52140.079068] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:08.22152 kern.info: [52140.294904] usb 3-4: reset full-speed USB device number 47 using xhci_hcd
2022-03-20T17:13:08.59958 kern.err: [52140.672994] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:08.82259 kern.err: [52140.896011] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:09.04556 kern.info: [52141.118855] usb 3-4: reset full-speed USB device number 47 using xhci_hcd
2022-03-20T17:13:09.04559 kern.warn: [52141.119069] usb 3-4: Device not responding to setup address.
2022-03-20T17:13:09.25151 kern.warn: [52141.325106] usb 3-4: Device not responding to setup address.
2022-03-20T17:13:09.45940 kern.err: [52141.532768] usb 3-4: device not accepting address 47, error -71
2022-03-20T17:13:09.57364 kern.info: [52141.646784] usb 3-4: reset full-speed USB device number 47 using xhci_hcd
2022-03-20T17:13:09.57367 kern.warn: [52141.646933] usb 3-4: Device not responding to setup address.
2022-03-20T17:13:09.77959 kern.warn: [52141.852925] usb 3-4: Device not responding to setup address.
2022-03-20T17:13:09.98750 kern.err: [52142.060794] usb 3-4: device not accepting address 47, error -71
2022-03-20T17:13:09.98752 kern.info: [52142.060899] usb 3-4: USB disconnect, device number 47
2022-03-20T17:13:10.12455 kern.info: [52142.197889] usb 3-4: new full-speed USB device number 48 using xhci_hcd
2022-03-20T17:13:10.23866 kern.err: [52142.311935] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:10.46245 kern.err: [52142.535796] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:10.68545 kern.info: [52142.758711] usb 3-4: new full-speed USB device number 49 using xhci_hcd
2022-03-20T17:13:10.79965 kern.err: [52142.872768] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:11.02154 kern.err: [52143.094875] usb 3-4: device descriptor read/64, error -71
2022-03-20T17:13:11.12357 kern.info: [52143.196924] usb usb3-port4: attempt power cycle
2022-03-20T17:13:11.51154 kern.info: [52143.584678] usb 3-4: new full-speed USB device number 50 using xhci_hcd
2022-03-20T17:13:11.52643 kern.info: [52143.600127] usb 3-4: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=12.11
2022-03-20T17:13:11.52648 kern.info: [52143.600133] usb 3-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
2022-03-20T17:13:11.52649 kern.info: [52143.600134] usb 3-4: Product: USB Receiver
2022-03-20T17:13:11.52649 kern.info: [52143.600136] usb 3-4: Manufacturer: Logitech
2022-03-20T17:13:11.53238 kern.info: [52143.606252] logitech-djreceiver 0003:046D:C52B.00D6: hiddev0,hidraw2: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:14.0-4/input2
2022-03-20T17:13:11.65943 kern.info: [52143.732881] logitech-hidpp-device 0003:046D:4055.00D7: HID++ 4.5 device connected.
PS: I also found fix for How to configure the power output of a USB (extHDD not able to start, most often)? - #6 by Animesh_Sahu trying the same, difference was that once pushed all way back, partial pull didn’t helped to get HDD detected, instead only partial (just enough to sink power) connection after fully disconnection gets it detected, then I can push it all the way back.