ReBAR Support

Atleast ya’ll have a bios firmware to mess around with :cry:

(13th Gen Intel user here :headstone: )

(Reposting/quoting Jean’s request in the slightly delusional hope that someone from framework would see it)

2 Likes

I guess 13 owners are forgotten

Here is AMD 13 integrated graphics:

And with Sonnet eGPU enclosure:

Would be nice if we could enable ReBAR support somehow.

1 Like

As a Linux user who successfully utilized an 11th gen + rx590 eGPU setup in the past, I would love to see if this 7840U would run well with an ARC A580 or RX6700XT these days. Thank you for posting this data.

Check what address range AMD preallocates to the TypeC PCIe ports. With Intel, this range is by far not big enough to fit any GPU’s full VRAM. That’d need to be solved first. Only then is it a question of whether the BIOS would support ReBar itself (which it actually might, according to some reports here and it detecting ReBar for some eGPUs in some applications, even though it is not working). Unknown if there are deep limitations of the TB/USB4 controllers.
Here the example of DevMgr, view resources by connection:
image

Only ~450 MiB per port. Such limits have been pretty standard. My desktop Maple Ridge controller uses ~740 MiB (but that is dual port). Still a far cry from 8 or more GiB.

(Note there will be 2 ranges. One is smaller and for IO, the larger one is the one for the actual VRAM)

1 Like

I did try a 5700xt, worked pretty well.

That’s good to hear

Maybe of interest here: in my response to FW13 AMD 7040 BIOS ReBAR support
I noticed that my 12th gen Framework now reserves 64 GiB for 3 of the 4 TB/PCIe ports. And the TB3 Alpine Ridge Controller in my Razer Core X gets actually assigned the full 64 GiB if attached directly (for a Goshen Ridge TB4 hub every downstream port gets a third curiously).
I sadly only have an old GPU in my Core X that does not support ReBar. So I cannot test anything else easily.

I am not 100% sure what affected this change, because at the time of my last screenshot here I am quite sure that it was only the ~450 MiB per port I showed there. I am running Windows 11 24H2 and the new drivers from the currently beta 13th&12th gen driver package. So probably one or the other allowed this.
The system may now be more ready for ReBar to actually work.
Although, like I said, the top left port is excluded from that and only gets the previous amount address space for some reason (Intel specs say 42 Bit addressing for the PCIe complex, 39 Bit addressing for physical memory, so at least 512GiB. No reason why it should run out of memory on 32 GiB RAM and 3x 64 GiB of USB4-PCIe reserved…)

1 Like

Hi! I just found out about this thread and would like to enable Resizable BAR on my Framework 13 Intel 12th gen laptop.

I used the tools to extract the offset and value of the variable to change and found the same values.

However, when I execute setup_var.efi, it fails to modify the variable, telling me the memory is write-protected.

Anyone had this and figured out how to disable this write protection?

Well, have you actually checked which prerequisites are already met and which are not. Specifically the PCIe address ranges for the TypeC PCIe port you are plugging your eGPU into and the TB controller of the eGPU.

Plus the report from GPU-Z on ReBar.

1 Like

Thanks! I looked at the Device manager, base on your example:

So 768 MiB.

Here is the report from GPU-Z:

As a reminder, this is a Framework 13 with Intel 12th gen.

I had this issue too (Framework 13 Intel 13th gen) seems that they flipped the bit on us :frowning: (thou tbh if Intel waved a gun in Framework’s face, I don’t blame Framework for blinking, for such a niche application it’s hardly worth them fighting intel over it, there are bigger more important battles to fight to benefit the rest of the Framework user base).

I am looking at hacking the bios flash chip and reworking it because damn it I want ReBAR, give me ReBAR or give me firey death :smiley: Just need to save up to upgrade to i7 and then I’ll have a second motherboard to sacrifice to the hardware hacking :smiley:

Mine is a 13th Gen Intel(R) Core™ i5-1340P, which supports 96GB of ram so definitely should have ReBAR support outside of the “no rebar for laptop” b/s c/o Intel (like with the no >2GB ram on Netbook and no more than 4GB ram on some NUCs both c/o Intel…)

edit rant asside, my lscpu

# lscpu 
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  16
  On-line CPU(s) list:   0-15
Vendor ID:               GenuineIntel
  Model name:            13th Gen Intel(R) Core(TM) i5-1340P
    CPU family:          6
    Model:               186
    Thread(s) per core:  2
    Core(s) per socket:  12
    Socket(s):           1
    Stepping:            2
    CPU(s) scaling MHz:  16%
    CPU max MHz:         4600.0000
    CPU min MHz:         400.0000
    BogoMIPS:            4377.60
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm cons
                         tant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm
                         2 ssse3 sdbg fma cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb 
                         ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clfl
                         ushopt clwb intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req hfi umip pk
                         u ospke waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize arch_lbr ibt flush_l1d arch_capabilities
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   448 KiB (12 instances)
  L1i:                   640 KiB (12 instances)
  L2:                    9 MiB (6 instances)
  L3:                    12 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-15
Vulnerabilities:         
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Not affected
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
  Srbds:                 Not affected
  Tsx async abort:       Not affected

I suppose there is a way to disable write protection because the firmware can be upgraded. Unfortunately, I don’t plan to order a new motherboard to play for now :slight_smile:

Yeah. With that limit, even if ReBar was supported from the BIOS, it could not actually do anything. So the big question is: do the other ports already have the 64 GiB reservation I was seeing (1260p, BIOS 3.08, Win11 24H2)? If not, what else is different on your 12th gen. And, if there is enough address space available, would ReBar just start working?
(the “auto” option mentioned a long time ago here, that is supposed to be the default on 12th gen could potentially handle it this way).

Do you suggest I try to plug the eGPU on each four ports of the laptop and see if the memory range changes?

Or could it be a limitation of that eGPU? It is a Razer Core X Chroma.

I might be wrong, but I don’t think rebar has to be a bios setting. In linux one can change the rebar size at the os level. I don’t know if windows has a way to change it at os level.

You can just look in the Device Manager, same as before if the other ports are less limited. Because for me 3 of the 4 ports have 64 GiB reserved. Then it would absolutely be worth trying those. Otherwise we need to find out what is different about your setup that you do not get that 64 GiB that would basically be a prerequisite for ReBar to do anything useful, even if it did work.

(and we still have the issue if ReBar helps performance anyway in this setup. ReBar just maps all of VRAM. That allows the CPU to write or read directly to it. But normally this is avoided. Because that is CPU intensive and cannot reach max. throughputs. Normally you instruct the GPU and it pulls the data out of system RAM or writes it to system RAM. That does not take any CPU time during the actual transfer and is the only way to reach max. transfer speeds.
The only reason to do the direct transfers is for short / small transfers where latency is important. And the GPU drivers seem to use heuristics per game and transfer size to switch between DMA transfers and direct transfers. With USB4/TB3 eGPUs having more latency just because of the additional stuff in between, those heuristics might be wrong and cause worse performance anyway)

I think there was sth. about that. But is always from the OS that this is decided. Or better the GPU drivers. And the BIOS just has to understand a command to do some work in actually changing the mapping + the hardware/BIOS must have all the other prerequisites to enable the remapping and the larger mappings in general.

It may be that Linux can circumvent the BIOS here if it has enough privileges to mess with hardware. Or it can just ignore what the BIOS gives as info, but give the command anyway. I have not looked deep enough into the OS / BIOS interaction. But most of the other prerequisites must already be true for that. If the PCIe switches do not have enough address space, then there is no use in this at all.

1 Like

I’d love to know how to do that, my observations in linux so far have been put in a card when ReBAR is off/not supported and it just complains bitterly about it (being unable to allocate the page) in dmesg when it tries to mount the eGPU (or GPU in desktop)

I have never done that. I just believe I have read sth. about so. else claiming to have done it. And then threw in what other vague things I know about PCIe initialization.

I just know that for my desktop PC, that does ReBar with natively attached GPU, it also showed up like that in GPU-Z via TB (with the controller being far too limited in address space).

But I have not pulled that same GPU out of my desktop again to recheck with my FW, now that I get enough address space there. That, together with the ReBar on/auto/off choice that was discussed here before and changing it manually from auto to on not doing anything, is why I figures it might just work once the address space is there. Which seems to depend on the OS, because I believe it changed just with a Windows update for me.

I just ordered an Intel Arc B580 and my eGPU enclosure, and then realized Framework doesn’t support ReBar? I think it’s pretty simple to add it, right? Just add another option in the UEFI/BIOS to control it. Hope we can get this feature implemented soon

1 Like