Ethernet Expansion Card

Patrick_Corey · October 1, 2022, 5:18pm

This Ethernet Expansion Cards do seem to be very delicate.

I can truly confirm that all my Ethernet switches within my lab environments are not the issue as I have many desktops, laptops, Dongles, MacBook Pros, Mac Minis, Servers, and docking stations working without any issue.

I am getting greater success to always do a reboot after I insert the Expansion Card to a new slot – I try to the rear USB C ports (right now as I am writing this post the card is on the left side back port).

I am seeing the card go offline and online periodically. I don’t have that experience with any other Ethernet device.

I have swapped the two cards between my original Frame work laptop (11th Generation I received in DEC 2021) and the new 12th Gen I received in September 2022. The experience is the same.

I am using POP!_OS 22.04 using kernel 5.19.0-76051900 on both systems.

No matter the configuration of the card or laptop the experience is the same where I see periodic drop and start of the network adapter.

On both laptops the Ethernet Expansion Card is recognized as USB Ethernet device (image attached from the Framework 11-gen as both cards do not work as well as the other laptop).

The PCI Ethernet device is my Sonnet SOLO10G SFP+ device – a network for a TrueNas server.

I am not sure if I am going to return 1 or both.

PhilS · October 28, 2022, 9:14pm

I’m running Ubuntu with kernel 5.19.0, only changes were adding the realtek UDEV rules.

Inserting the card is like roulette, sometimes it just doesn’t work:

[  +0.331330] usb 2-2: new SuperSpeed USB device number 62 using xhci_hcd
[  +0.020732] usb 2-2: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.04
[  +0.000010] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[  +0.000004] usb 2-2: Product: USB 10/100/1G/2.5G LAN
[  +0.000002] usb 2-2: Manufacturer: Realtek
[  +0.000002] usb 2-2: SerialNumber: 4013000001
[  +0.023456] cdc_ncm 2-2:2.0: MAC-Address: 9c:bf:0d:00:05:30
[  +0.000009] cdc_ncm 2-2:2.0: setting rx_max = 16384
[  +0.000037] cdc_ncm 2-2:2.0: setting tx_max = 16384
[  +0.000473] cdc_ncm 2-2:2.0 eth0: register 'cdc_ncm' at usb-0000:00:0d.0-2, CDC NCM, 9c:bf:0d:00:05:30
[  +0.590028] usb 2-2: Disable of device-initiated U1 failed.
[  +0.007090] usb 2-2: Disable of device-initiated U2 failed.
[  +0.000030] cdc_ncm 2-2:2.0 eth0: unregister 'cdc_ncm' usb-0000:00:0d.0-2, CDC NCM
[  +0.190475] usb 2-2: Set SEL for device-initiated U1 failed.
[  +0.007047] usb 2-2: Set SEL for device-initiated U2 failed.
[  +1.828562] usb 2-2: USB disconnect, device number 62
[  +2.476044] usb usb3-port3: Cannot enable. Maybe the USB cable is bad?
[  +2.843934] usb usb3-port3: Cannot enable. Maybe the USB cable is bad?
[  +0.000114] usb usb3-port3: attempt power cycle
[  +2.711968] usb usb3-port3: Cannot enable. Maybe the USB cable is bad?
[  +2.643836] usb usb3-port3: Cannot enable. Maybe the USB cable is bad?
[  +0.000119] usb usb3-port3: unable to enumerate USB device

Eventually with enough port sex:

[Oct28 14:06] usb 2-2: new SuperSpeed USB device number 63 using xhci_hcd
[  +0.020725] usb 2-2: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.04
[  +0.000008] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[  +0.000003] usb 2-2: Product: USB 10/100/1G/2.5G LAN
[  +0.000002] usb 2-2: Manufacturer: Realtek
[  +0.000002] usb 2-2: SerialNumber: 4013000001
[  +0.023476] cdc_ncm 2-2:2.0: MAC-Address: 9c:bf:0d:00:05:30
[  +0.000007] cdc_ncm 2-2:2.0: setting rx_max = 16384
[  +0.000035] cdc_ncm 2-2:2.0: setting tx_max = 16384
[  +0.000413] cdc_ncm 2-2:2.0 eth0: register 'cdc_ncm' at usb-0000:00:0d.0-2, CDC NCM, 9c:bf:0d:00:05:30
[  +0.003163] cdc_ncm 2-2:2.0 eth0: unregister 'cdc_ncm' usb-0000:00:0d.0-2, CDC NCM
[  +0.164569] usb 2-2: reset SuperSpeed USB device number 63 using xhci_hcd
[  +0.047373] r8152 2-2:1.0: load rtl8156b-2 v1 04/15/21 successfully
[  +0.037585] r8152 2-2:1.0 eth0: v1.12.13
[  +0.587093] r8152 2-2:1.0 enx9cbf0d000530: renamed from eth0
[  +3.291778] IPv6: ADDRCONF(NETDEV_CHANGE): enx9cbf0d000530: link becomes ready
[  +0.000346] r8152 2-2:1.0 enx9cbf0d000530: carrier on

Once it works, I’m getting good performance with iperf3 as client:

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   283 MBytes  2.37 Gbits/sec    0    624 KBytes       
[  5]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    949 KBytes       
[  5]   2.00-3.00   sec   281 MBytes  2.36 Gbits/sec    0    949 KBytes       
[  5]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    996 KBytes       
[  5]   4.00-5.00   sec   281 MBytes  2.36 Gbits/sec    0    996 KBytes       
[  5]   5.00-6.00   sec   280 MBytes  2.35 Gbits/sec    0    996 KBytes       
[  5]   6.00-7.00   sec   281 MBytes  2.36 Gbits/sec    0    996 KBytes       
[  5]   7.00-8.00   sec   280 MBytes  2.35 Gbits/sec    0    996 KBytes       
[  5]   8.00-9.00   sec   279 MBytes  2.34 Gbits/sec    0   1.09 MBytes       
[  5]   9.00-10.00  sec   281 MBytes  2.36 Gbits/sec    0   1.09 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.74 GBytes  2.35 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  2.74 GBytes  2.35 Gbits/sec                  receiver

However, when it is a server, the retransmits really screw up performance:

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  89.3 MBytes   749 Mbits/sec  869   18.4 KBytes       
[  4]   1.00-2.00   sec  89.6 MBytes   752 Mbits/sec  904   19.8 KBytes       
[  4]   2.00-3.00   sec  92.3 MBytes   774 Mbits/sec  946   17.0 KBytes       
[  4]   3.00-4.00   sec  92.3 MBytes   775 Mbits/sec  931   19.8 KBytes       
[  4]   4.00-5.00   sec  92.9 MBytes   779 Mbits/sec  876   19.8 KBytes       
[  4]   5.00-6.00   sec  92.4 MBytes   775 Mbits/sec  935   19.8 KBytes       
[  4]   6.00-7.00   sec  93.0 MBytes   780 Mbits/sec  967   19.8 KBytes       
[  4]   7.00-8.00   sec  90.9 MBytes   763 Mbits/sec  899   19.8 KBytes       
[  4]   8.00-9.00   sec  87.6 MBytes   734 Mbits/sec  929   17.0 KBytes       
[  4]   9.00-10.00  sec  78.0 MBytes   655 Mbits/sec  756   19.8 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   898 MBytes   754 Mbits/sec  9012             sender
[  4]   0.00-10.00  sec   898 MBytes   753 Mbits/sec                  receiver

I’ve tested this against multiple machines each on a 10G link, which all achieve 9.39Gb/s between one another.

Also, once I disconnect the ethernet cable and reconnect, it will never link again until I reset the card.

PhilS · October 28, 2022, 9:42pm

Update: Once the card is working hard after about 5 minutes, it will disconnect, this seems to repeat over and over. I think framework made a dud. I’d advise people not order this until these problems are fixed, I think I’m going to try to return mine.

[Oct28 14:39] usb 2-2: USB disconnect, device number 65
[  +0.000269] xhci_hcd 0000:00:0d.0: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[  +0.335803] usb 2-2: new SuperSpeed USB device number 66 using xhci_hcd
[  +0.020433] usb 2-2: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.04
[  +0.000013] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=6
[  +0.000003] usb 2-2: Product: USB 10/100/1G/2.5G LAN
[  +0.000003] usb 2-2: Manufacturer: Realtek
[  +0.000002] usb 2-2: SerialNumber: 4013000001
[  +0.024357] cdc_ncm 2-2:2.0: MAC-Address: 9c:bf:0d:00:05:30
[  +0.000007] cdc_ncm 2-2:2.0: setting rx_max = 16384
[  +0.000038] cdc_ncm 2-2:2.0: setting tx_max = 16384
[  +0.000500] cdc_ncm 2-2:2.0 eth0: register 'cdc_ncm' at usb-0000:00:0d.0-2, CDC NCM, 9c:bf:0d:00:05:30
[  +0.004007] cdc_ncm 2-2:2.0 eth0: unregister 'cdc_ncm' usb-0000:00:0d.0-2, CDC NCM
[  +0.162974] usb 2-2: reset SuperSpeed USB device number 66 using xhci_hcd
[  +0.024034] r8152 2-2:1.0: load rtl8156b-2 v1 04/15/21 successfully
[  +0.036704] r8152 2-2:1.0 eth0: v1.12.13
[  +0.578218] r8152 2-2:1.0 enx9cbf0d000530: renamed from eth0
[  +3.339795] IPv6: ADDRCONF(NETDEV_CHANGE): enx9cbf0d000530: link becomes ready
[  +0.000324] r8152 2-2:1.0 enx9cbf0d000530: carrier on