Ethernet expansion card photos, quick start guide, benchmarking

Fraoch · September 13, 2022, 8:20pm

I’ve been lucky enough to test an early production version of the Ethernet expansion card and wanted to pass along my findings.

Photos

Flush with the top of the input lid and extending out the side a little less than the Framework right-angle charging cable, also note green LINK LED on the left and amber ACT LED on the right:

Extends down by less than the foot height on the underside:

Quick Start Guide: Linux

Plug in Ethernet expansion card.
Plug in Ethernet cable.

Yes, that’s it.

The RTL8156 driver is built into the Linux kernel and should be present in every kernel since (at least) 5.8 - kernels that fully support the Framework laptop are newer than that, so if your Framework laptop runs 5.13+ the Ethernet card should be supported out of the box.

Updated drivers are here but are included with kernel 5.17+.

Quick Start Guide: Windows 10

Plug in Ethernet expansion card.
Plug in Ethernet cable.

Windows 10 does seem to support it out of the box.

I was unable to test on Windows 11.

Benchmarking: Linux 2.5 GbE

I used iperf2 between this and a Realtek RTL8125-equipped iperf server connected through a QNAP QSW-1105-5T 2.5 GbE switch. The RTL8125 is the PCIe-based equivalent of the USB-based RTL8156. The test was repeated 3 times to ensure consistency and averaged results.

mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size: 1.41 MByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 39066 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  2.74 GBytes  2.36 Gbits/sec
mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size: 1.67 MByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 39068 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  2.74 GBytes  2.36 Gbits/sec
mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size: 1.80 MByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 39070 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  2.74 GBytes  2.36 Gbits/sec

No need to average these results, it’s a consistent 2.36 Gbit/s. This is nearly identical to the results of another RTL8156 card I have and it seems 2.36 Gbit/s is “wire speed” for 2.5 GbE through a switch.

Benchmarking: Windows 2.5 GbE

Same equipment, same test.

The drivers were the built-in Windows drivers.

mark@Sauron:~$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 50176 (peer 2.1.7-beta)
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  2.77 GBytes  2.37 Gbits/sec
[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 50178 (peer 2.1.7-beta)
[  4]  0.0-10.0 sec  2.76 GBytes  2.37 Gbits/sec
[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 50179 (peer 2.1.7-beta)
[  4]  0.0-10.0 sec  2.76 GBytes  2.37 Gbits/sec

Perfect, even better than Linux. Also an indication this is 2.5 GbE wire speed.

But - I installed the latest drivers and regretted it:

[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 49866 (peer 2.1.7-beta)
[  4]  0.0-10.0 sec  2.33 GBytes  1.99 Gbits/sec
[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 49867 (peer 2.1.7-beta)
[  4]  0.0-10.0 sec  1.80 GBytes  1.55 Gbits/sec
[  4] local 192.168.1.77 port 5001 connected with 192.168.1.55 port 49869 (peer 2.1.7-beta)
[  4]  0.0-10.0 sec  2.33 GBytes  2.00 Gbits/sec

I can’t explain these results but it would probably be best to avoid the latest driver, wait for an updated driver and be prepared to roll back should you see this.

Benchmarking: Linux & Windows 1 GbE

All tests as before, but through a 1 GbE TP-Link TL-SG2008P:

mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size: 1012 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 45278 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   944 Mbits/sec
mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size:  850 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 45280 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   943 Mbits/sec
mark@SamwiseGamgee:~$ iperf -c 192.168.1.77
------------------------------------------------------------
Client connecting to 192.168.1.77, TCP port 5001
TCP window size:  799 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.55 port 45282 connected with 192.168.1.77 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   943 Mbits/sec

945 Mb/s is considered wire speed for 1 GbE. No problems here.

Windows results were identical.

USB Latency Check

Pinging to my router over the 2.5 GbE switch:

mark@SamwiseGamgee:~$ ping -c 10 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=63 time=0.931 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=63 time=1.01 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=63 time=0.949 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=63 time=0.800 ms
64 bytes from 192.168.1.1: icmp_seq=5 ttl=63 time=1.25 ms
64 bytes from 192.168.1.1: icmp_seq=6 ttl=63 time=0.982 ms
64 bytes from 192.168.1.1: icmp_seq=7 ttl=63 time=0.965 ms
64 bytes from 192.168.1.1: icmp_seq=8 ttl=63 time=1.14 ms
64 bytes from 192.168.1.1: icmp_seq=9 ttl=63 time=1.00 ms
64 bytes from 192.168.1.1: icmp_seq=10 ttl=63 time=0.907 ms

--- 192.168.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9013ms
rtt min/avg/max/mdev = 0.800/0.992/1.245/0.116 ms

Results for 1 GbE were nearly identical.

Comparing this to the best-case-scenario PCIe RTL8125 through the same chain of networking equipment:

mark@Sauron:~$ ping -c 10 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=63 time=0.699 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=63 time=0.530 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=63 time=0.659 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=63 time=0.556 ms
64 bytes from 192.168.1.1: icmp_seq=5 ttl=63 time=0.557 ms
64 bytes from 192.168.1.1: icmp_seq=6 ttl=63 time=0.669 ms
64 bytes from 192.168.1.1: icmp_seq=7 ttl=63 time=0.716 ms
64 bytes from 192.168.1.1: icmp_seq=8 ttl=63 time=0.978 ms
64 bytes from 192.168.1.1: icmp_seq=9 ttl=63 time=0.550 ms
64 bytes from 192.168.1.1: icmp_seq=10 ttl=63 time=0.426 ms

--- 192.168.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9172ms
rtt min/avg/max/mdev = 0.426/0.634/0.978/0.142 ms

The additional latency added by the USB connection is negligible: 0.36 ms.

Switch Compatibility Check

I also tested this attached to a QNAP QSW-M2108-2S and the results were identical. I did not have any switch compatibility issues although I only had a sample size of 2. Donations accepted!

Ethernet Cable Compatibility Check

I tested the card with a CAT5e cable and a CAT6 cable. Both worked perfectly. I even tried a CAT5 cable which should not be expected to work at 2.5 GbE but it did without an issue.

Conclusion

In Linux, this works great for me in both 2.5 GbE and 1 GbE modes. At 1 GbE, there should be no issues on either Windows or Linux.

At 2.5 GbE, there should be no issues on Linux. Windows numbers could be improved at 2.5 GbE - updated drivers should help. The drivers included with Windows should work well though.

nez · September 14, 2022, 10:48am

Any data on the power draw of the module? I know the HDMI plug pulled power even when not in use.

Thanks for running this, and great timing Framework. I’m about to troubleshoot a local community building and was loathe to try and do it with my locked down work machine.

Fraoch · September 14, 2022, 12:24pm

I’ll have to research how to do that, never done it before.

Fraoch · September 15, 2022, 7:25pm

OK I measured power consumption with powerTOP.

This is quite tricky, but essentially the baseline power consumption on battery at idle with nothing but USB-C cards (or no cards) installed was:

3.59 W minimum

With the card installed at idle:

6.3 W minimum

So 2.71 W idle.

With the card installed doing heavy 2.5 GbE transfers:

12.4 W maximum

So 8.81 W when being pushed hard.

I also forgot to mention - there is very little heat output. After a series of 2.5 GbE tests my IR thermometer (aimed right at the controller IC) measured 33.3°C in a warm room of 25.3°C, that’s a delta of only 8°C over ambient. On the case itself the warmest component was the steel network port casing, it measured 29.6°C so it obviously provided some heatsinking but was barely warm to the touch.

NM64 · September 16, 2022, 1:46am

I don’t suppose the ethernet card could be plugged in upside down so as to lift the laptop up a bit more in the back?

…though, without a second card on the opposite side, it could be a bit un-even (now we just need a similar “extra large” expansion card with multiple ports, one of which being a type C specifically for power since that, plus 2.5g ethernet, would make for an ideal docking scenario)

If you happened to have a watt meter then one can measure it by simply disconnecting the battery.

I in particular am interested in how much power it would draw while the laptop is shut down.

Fraoch · September 16, 2022, 11:47am

The channel on this card guiding it into the laptop expansion card rails is offset and can only go in one way.

nez · September 20, 2022, 7:41pm

Wow that’s a lot of juice! Thanks for testing. I’m purchasing one regardless, but I’ll avoid having it plugged in if its not in use. Eventually I need to figure out how to selectively deactivate USB-C ports so I can keep the hardware plugged in when away from home without draining the battery.

Fraoch · September 20, 2022, 7:45pm

I could be wrong. It’s very hard to measure power consumption with powerTOP, it’s drifting all over the place. I tried to measure it over a period of several minutes, noting the reading whenever it changed (looks like every 30 seconds or so). I used the minimum reading over that time frame.

nez · September 21, 2022, 10:08pm

Just got mine today and it worked for thirty seconds, then stopped.

Known good cable (its stolen from my gaming rig). When I plug in, lights flicker for a second then go off.

Linux Mint 20.3
The device is visible in ifconfig:
enx9cbf0d00024b: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 9c:bf:0d:00:02:4b txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

I checked TLP and I already turned off USB disabling and no change in behavior after plugging in (TLP does also see the device):

Bus 002 Device 006 ID 0bda:8156 control = auto, autosuspend_delay_ms = 2000 – Realtek Semiconductor Corp. USB 10/100/1G/2.5G LAN (cdc_ncm)

Any insight?

Fraoch · September 21, 2022, 10:16pm

What kernel? It would be 5.13 if I recall correctly, and there are updated drivers included in 5.17.

Try the updated drivers here: Realtek USB FE / GBE / 2.5G / 5G Ethernet Family Controller Software - REALTEK They’re very easy to install.

Before doing this, take a Timeshift backup to ensure you can walk back if it goes really badly.

nez · September 21, 2022, 10:43pm

That worked!!! I was working through the list, but since it did in fact work for a period there I discounted drivers.

(And yes, still getting used to cutting edge technology, and not recycled parts I find on the side of the road )

Thank you!!

Fraoch · September 21, 2022, 11:10pm

Whew, glad that helped!

mackncheesiest · September 22, 2022, 10:05pm

Hmm, my card doesn’t even show up as a device specifically if I attach it to either of the “closer to the front” 2/4 expansion ports (Fedora 36, kernel 5.19.9). No interface shows up under ip link / ifconfig, and if I attach to a 1 GbE switch, nothing happens.

Excerpt from journalctl -r

Sep 22 14:40:41 host kernel: usb usb2-port3: config error

...
Sep 22 14:40:21 host kernel: usb usb2-port3: Cannot enable. Maybe the USB cable is bad?
...
Sep 22 14:40:17 host kernel: usb 2-3: device not accepting address 9, error -71
Sep 22 14:40:17 host kernel: usb 2-3: Device not responding to setup address.
Sep 22 14:40:17 host kernel: usb 2-3: Device not responding to setup address.
Sep 22 14:40:15 host kernel: usb usb2-port3: attempt power cycle

Perhaps I have a hardware issue on those ports that other expansion cards haven’t made apparent before. With the other two ports, the link shows up, but no luck actually connecting to my router after testing with a few different cables.

Sep 22 14:56:56 host mtp-probe[7931]: bus: 2, device: 36 was not an MTP device
Sep 22 14:56:56 host mtp-probe[7931]: checking bus 2, device 36: "/sys/devices/pci0000:00/0000:00:0d.0/usb2/2-4"
Sep 22 14:56:56 host NetworkManager[1468]: <info>  [1663883816.3231] settings: (enp0s13f0u4c2): created default wired connection 'Wired connection 1'
Sep 22 14:56:56 host NetworkManager[1468]: <info>  [1663883816.3197] device (enp0s13f0u4c2): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Sep 22 14:56:56 host NetworkManager[1468]: <info>  [1663883816.3096] device (eth0): interface index 10 renamed iface from 'eth0' to 'enp0s13f0u4c2'
Sep 22 14:56:56 host kernel: cdc_ncm 2-4:2.0 enp0s13f0u4c2: renamed from eth0
...
Sep 22 14:56:55 host mtp-probe[7882]: bus: 2, device: 36 was not an MTP device
Sep 22 14:56:55 host mtp-probe[7882]: checking bus 2, device 36: "/sys/devices/pci0000:00/0000:00:0d.0/usb2/2-4"
Sep 22 14:56:55 host NetworkManager[1468]: <info>  [1663883815.6871] manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/10)
Sep 22 14:56:55 host kernel: cdc_ncm 2-4:2.0 eth0: register 'cdc_ncm' at usb-0000:00:0d.0-4, CDC NCM, 9c:bf:0d:00:02:6f
Sep 22 14:56:55 host kernel: cdc_ncm 2-4:2.0: setting tx_max = 16384
Sep 22 14:56:55 host kernel: cdc_ncm 2-4:2.0: setting rx_max = 16384
Sep 22 14:56:55 host kernel: cdc_ncm 2-4:2.0: MAC-Address: 9c:bf:0d:00:02:6f
Sep 22 14:56:55 host kernel: usb 2-4: SerialNumber: 4013000001
Sep 22 14:56:55 host kernel: usb 2-4: Manufacturer: Realtek
Sep 22 14:56:55 host kernel: usb 2-4: Product: USB 10/100/1G/2.5G LAN
Sep 22 14:56:55 host kernel: usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=6
Sep 22 14:56:55 host kernel: usb 2-4: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.04
Sep 22 14:56:55 host kernel: usb 2-4: new SuperSpeed USB device number 36 using xhci_hcd

Anyone have thoughts on things to test?

nez · September 23, 2022, 3:56pm

It works normally on the other two positions? And those front two slots work for other devices like HDMI or USB-A?

mackncheesiest · September 23, 2022, 5:49pm

So in the back two positions, the adapter is detected and actually registered as an ethernet device. It even gets an IPv6 address (probably through stateless autoconfiguration), but IPv4 DHCP never finishes. In the logs, it looks like the device basically keeps resetting but over a longer time window.

In the front two positions, the adapter isn’t properly probed as a USB device at all (the case shown in the first set of logs). The kernel just keeps power cycling it and attempting to probe it, but it eventually gives up. USB-A/C/1 TB storage module generally work fine in these front two ports. DisplayPort tends to give more issues in the front ports than the back ones, but I’ve also never really been able to get my dual-monitor setup to consistently work even if I only use the back ports (always have assumed it’s the cables or something because the kernel logs report link training issues).

For this ethernet adapter, I’ve tried enabling a debug kernel (dnf install kernel-debug), turning off secure boot to disable kernel lockdown, and enabling dynamic debug for the r8152 driver (echo 'file drivers/net/usb/r8152.c +p'>/sys/kernel/debug/dynamic_debug/control) to see if anything else shows in dmesg, but no new insights so far. Not sure if I need to be actually building a custom kernel image with #define DEBUG set in that driver file to actually get the dev_dbg statements. Also tried entirely disabling TLP in case there was some kind of USB power setting that it was messing with that caused issues.

To make things more confusing, in Windows basically all of my devices work. This ethernet adapter works out-of-the-box in all 4 ports, my DisplayPort connections are more reliable, etc. Are the drivers just more resilient in Windows perhaps?

Edit: I can also move this to a thread in the community support area if we want to keep this thread a bit more focused

mackncheesiest · September 23, 2022, 6:08pm

And as one more update: in a Fedora 36 live USB (which boots with 5.17.5), the back two ports just work entirely fine, but the front two still have issues with probing & power cycling the device when it’s plugged in.

Shirley_Dulcey · September 24, 2022, 4:59am

Is that idle power measured with an Ethernet cable plugged in and attached to a switch, or with just the module by itself? The numbers might well be different if the adapter goes into a low power mode if it’s not connected to anything.

Shirley_Dulcey · September 24, 2022, 5:53am

That higher power consumption when doing a 2.5 GBe transfer probably isn’t all going to the Ethernet adapter. The CPU is also working harder. It’s likely that’s where most of the additional power is going, not the Ethernet adapter.

Shirley_Dulcey · September 24, 2022, 6:00am

I’m not very worried about the in-use power numbers, whatever they may be, because connected to Ethernet but running on batteries is likely to be a rare use case. Most of the time when you’re connected to a wired network it will also be easy to plug in the charger. Standby power with the adapter plugged in matters more, because disconnecting the power and LAN and packing up the laptop is something that many of us will do.

I received mine today. Working perfectly so far, and it doesn’t even get warm to the touch in normal use. Can’t try it on a 2.5 GBe network because I don’t have any other equipment that runs at that speed, and I’d probably have to pull new cable to upgrade my house to higher speeds. It’s currently a mix of Cat 5 and Cat 5e in the walls; the only Cat 6 is some of the whips.

Jeef · September 24, 2022, 3:02pm

The in use power numbers do concern me a bit. My main reason for ordering the module is because when I’m connecting to Ethernet on a laptop I’m walking around to racks and switches to troubleshoot vs if stationary I’m using a dock