I’m having a bit of trouble understanding what your saying.
Do you mean that upon/after boot whether your device is plugged/unpluggd it’s stable
but later during the session ( without rebooting ) you plug/unplug your adapter, that’s when it becomes unstable?