[GUIDE] Manjaro, Kernel 5.18+, eGPU, Nvidia

Framework Laptop 13 Linux
1

Hello everyone,

I’ve been fighting and struggling big time to get my 3090 in my Razer Core X Chroma working… at all in Manjaro. Works perfect in Windows 11. So as I’ve been trying to work through roadblock after roadblock, I figured I’d just write everything down with the end goal being a decent guide for everyone else. Hopefully. Otherwise it’ll end up a bunch of random thoughts. XD

Update: I now have it working.

inxi -G 
Graphics:
  Device-1: Intel Alder Lake-P Integrated Graphics driver: i915 v: kernel
  Device-2: NVIDIA GA102 [GeForce RTX 3090] driver: nvidia v: 515.76
  Device-3: Realtek Laptop Camera type: USB driver: uvcvideo
  Display: x11 server: X.org v: 1.21.1.4 with: Xwayland v: 22.1.3 driver:
    X: loaded: modesetting,nvidia unloaded: nouveau gpu: i915 resolution:
    1: 1920x1080~144Hz 2: 2256x1504~60Hz
  OpenGL: renderer: Mesa Intel Graphics (ADL GT2) v: 4.6 Mesa 22.1.7
nvidia-smi 
Wed Sep 28 10:55:12 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.76       Driver Version: 515.76       CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:7F:00.0  On |                  N/A |
|  0%   54C    P8    42W / 420W |     46MiB / 24576MiB |     23%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1761      G   /usr/lib/Xorg                      45MiB |
+-----------------------------------------------------------------------------+

NVIDIA Open Drivers
For some reason using the normal proprietary drivers would never work and led to just black screens all the time. The only way I could get the NVIDIA GPU to work was to use the nvidia-open drivers as suggested by @Mapleleaf.

In Manjaro, they are not officially supported yet however you can find them in Gitlab. There are two packages that you need to install:

These can be built / installed just like they are AUR packages.

Kernel 5.18+
As the 12th generation processor works better on kernels later than 5.18, I’m currently running 5.19. Every time I would install the Nvidia drivers, I would reboot and then get a black screen unable to switch TTYs. If I booted to run-level 3 and executed inxi -G the driver for the Nvidia GPU would say “NA”.

This issue came down to something called Intel’s Indirect Branching Tracking. This feature only exists starting with kernel 5.18. Kernels before will not have this issue. The other way to fix this issue is by disabling this feature in the kernel with ibt=off.

This is not needed if using the nvidia-open drivers.

This solution was found on forum.manjaro.org.

Wayland
For lots of reasons, I prefer Wayland. I was a bit confused when I tried to boot with the Nvidia driver only to find Wayland was unavailable. As it turns out GDM’s udev rules disable wayland if certain criteria is met, some of which include the Nvidia driver. You can modify the /usr/lib/udev/rules.d/61-gdm.rules and still allow Wayland to start.

It should be noted that where I found this solution also mentioned needing to include the nvidia-drm.modeset=1 kernel parameter. I am not sure if this is still needed or not.

This solution was found on medium.com.

So while this is possible, I probably do not recommend this yet. I tried it and the performance wasn’t great. Normally with my favorite valley benchmark, the 3090 gets 200+ FPS. However, running under Wayland (which already is a bit jerky just on the desktop) was only getting 30-40 FPS.

Razor Core X Chroma
The unit comes with multiple USB ports and a built-in 1GB NIC. For me, the USB ports had always worked without issue. The NIC though would only be sometime detected. The trick seems to be to leave it unplugged until GRUB loads and plug it in then. That made Manjaro see the NIC and allowed me to use it.

It should be noted that the card I bought needed 3 8-pin power cables and the unit only comes with 2. I bought an upgraded modular power supply as well as an extra cable kit to get 3 dedicated cables for the 8-pin connectors. This required me to either make / buy a bracket so that I could put in the new PSU and a more quiet fan.

5 Likes
2

I am contemplating an external GPU. My Framework laptop should be shipping soon. Leaning more toward Ubuntu because that is what I am familiar with. Thinking an RTX 3060 ti and the non-chroma core enclosure. I am interested in your progress.

3

Hopefully i can work out all the kinks so by the time you get to it, there wont be a lot to figure out.

4

My eGPU (GeForce RTX 3060 Ti + Razer Core X) is working fine, and I’m using an Artix (Arch without Systemd).
To get it to work I had to install (with pacman) the nvidia-open drivers instead of the nvidia. Maybe this is the problem for you?
As soon as the eGPU is plugged, nvidia-smi reports correctly the resource. (hotplug is fine, but hot-unplug is disastrous).
I also added this file, /etc/X11/xorg.conf.d/80-egpu-primary-igpu-offload.conf:

Section "Device"
    Identifier "Device0"
    Driver     "modesetting"
EndSection

Section "Device"
    Identifier "Device1"
    Driver     "nvidia"
    BusID      "PCI:130:0:0"                 # Edit according to lspci, translate from hex to decimal.
    Option     "AllowExternalGpus" "True"    # Required for proprietary NVIDIA driver.
EndSection

Moreover, to launch e.g. the game 0ad using GPU acceleration, I need to use these environment variables:

$ __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia 0ad

I set a convenient alias primnv and then I just have to do:

$ primnv 0ad

For all the games I tried, the environment variables above were the right ones to use, but weirdly enough for glxgears I needed to use a different one:

$ DRI_PRIME=1 glxgears

Details about my config:
I’m using Xorg, my kernel is 5.19.4, my Xorg driver is modesetting (NOT intel).

1 Like
5

That’s interesting. I didn’t even know those drivers existed. Looks like Manjaro doesn’t support using them yet.

6

It looks like you can install AUR packages on Manjaro too:
https://forum.manjaro.org/t/how-can-i-add-archlinux-repositories/29594/2

7

I think the root cause of why it works with nvidia-open but not nvidia on Arch-based distribs is this:
https://bugs.archlinux.org/task/74886

A bug about Intel IBT support was fixed (patched specific for Arch) in the kernel and nvidia-open packages by Arch maintainers, but of course it cannot be fixed in the closed-source proprietary nvidia drivers.

1 Like
8

You can supply ibt=off that I talk about in my original post to disable ibt and allow the regular Nvidia drivers to load.

1 Like
9

Makese sense. On Manjaro (and maybe yours to?) there is a prime-run command that gets installed which is a bash script that does the same thing.

1 Like
10

@nomb85 You’re right, I did not check but indeed the prime-run command from the nvidia-prime package does exactly what my alias does.

I was reluctant to rely on a package because, as I use nvidia-open, I had to maintain two aliases anyway, one for DRI thing and the other for the PRIME RENDER OFFLOAD thing, depending on which program I wanted to run.

By the way I still don’t have any idea of how to guess which programs use which (except by trying) and why they do so.

11

My understanding is:

  • DRI_PRIME = opengl
  • prime-run = vulkan / non-opengl
1 Like
12

Thanks! That makes sense but it’s kind of surprising, then, that almost all of the “nice” open-source games don’t use OpenGL… (0AD, Zero-K, Extreme TuxRacer, …)

1 Like
13

In regards to the pci device for the xorg config, is this the gpu device id or the thunderbolt controller it is connected it? Running a radeon 5700xt and cant seem to get your instructions working.

14

@sky It’s the GPU. What does the command lspci gives?

But wait, these instructions were for a NVIDIA GPU, so it will probably need to be adapted a bit for AMD.

15

@Mapleleaf Hey man thanks a lot for posting that udev config information. I was struggling for awhile to get everything setup properly. I install Manjaro then used that config and it worked perfectly. Why isn’t this just auto enabled is wild, but hey its working thanks.

1 Like
16

@Ryein_Goddard Thanks a lot!! I’m really happy to be helpful.
(But hey, a nitpick, it’s a Xorg conf, not udev, right?)

2 Likes
17

@Mapleleaf haha yeah you are right. I must be tired.

1 Like
18

By the way, it looks like ibt=off is no longer required for (non-open) Nvidia drivers (since 30 March 2023 already):

For nvidia-open drivers it was already the case before.

19

I just wish Manjaro officially supported nvidia-open and we didn’t have to pull it from their Gitlab.