Some values persist, some values reset on boot, some reset on failure. Changing some values also causes the bios to change other values.
There’s an initialization vector that reset much of the CBS values to defaults too, although it must either be loading the bios stored values from elsewhere or it knows to skip it.
I’ve tried altering just about everything that seems like it would adjust vram size with no luck. It’s likely the bios code itself needs to be patched since it keeps resetting UMA_SPECIFIED to UMA_AUTO on boot. I don’t know where auto and game optimized pull their ram settings from either, although I believe I’ve had a few boots where auto booted with 4GB vram instead of 512MB. I’m not exploring this too methodically yet though, I was hoping for an easy win.
I did notice some variables in the ConIn section as well as a vga section are changing as I’m messing with settings, so I may possibly have some other settings to explore. I’m not too worried since the bios seems to reset to factory defaults if it’s the unable to boot.
This would be huge as I really don’t want to go back to windows at all to do any machine learning.
Also one of the reason I bought a lot of RAM.
I don’t think we have a way to vote with our wallet for this feature request (like a feature bounty) but I would definitely put some money towards this if it meant we could get it sooner rather than later.
Hopefully we get some more info from someone on the FW team.
If you want to invest time, money and or dev-work then that would be far more effective in developing the software to work with dynamic host memory allocation instead of waiting for others to support the legacy workaround in firmware.
There has even been some movement for example in pytorch of developers realizing that this would be the cheapest way to get very memory intensive models to run and going over why the plugin-hack does not work and what needs to change to get the host-allocation to work for everything.
Also the way how you’d get non-power-of-2 memory sizes usable for the iGPU and potentially more than half of the system memory.
While I agree this is the better long term solution, it is not the most practical right now.
By fixing this one issue several dozen pieces of software would just work, no more work needed. If we focused efforts on all the different libraries and apps we want we would need to fix a couple dozen before we put a dent on perception on how useful fixing this is.
I am also guessing that most people who have this issue just go back to windows whenever they want to do some testing or experiment (or playing the games they cant with this).
On the game front it might not be noticeable because most legacy games (where this can’t be patched) do not use > 4Gb or vRAM.
I started looking at some of the info with efivar, I think that some of the changes are overwritten by the bios because it reloads the settings from its -db setting, so in order for changes to stick changes might need to be done on the variable and the -db setting too?
Just wanted to chime in and say I also want this feature, a few things I’ve used that were lazily ported and designed just for dGPUs will crash if they encounter too little VRAM. To the people saying this is a “hack” and not a best practice, welcome to firmware workarounds, that’s more or less what they all are, haha. Back in the BIOS days there were all sorts of flags to let a handful of programs or a single OS work right on a board, and there’s still all kinds of legacy cruft that can be enabled in most BIOS as workarounds for people with very specific needs from their workflows. This feature would be useful to some of us, and its existence would have no impact whatsoever on everyone else who only does things the “right” way.
I still think having a shim program that dynamically requests the ram and then launches the problematic thing with that is better than just statically walling of a ton of ram for them.
But something like that would definitely help a lot even if it is a hack.
While I understand the other system would be better and more flexible in many cases, having something in the firmware means never having to worry if in five years someone will have updated this hypothetical program to work with the latest OSes. Furthermore it makes sure it actually works on more OSes than just whatever the current version of Windows is, which in my experience is usually what ends up happening with useful tools like that. Since this is a problem with the way the software interfaces with the hardware itself, it’s good to have a fallback solution as close to the metal as possible. A hypothetical shim program would be great, nothing is stopping anyone from developing that, but this thread is about wanting a UEFI based option, specifically.
I just want to mention that I tried this on Lenovo ThinkPad T14s (Ryzen7 PRO 7840U, 32 GB RAM), and their BIOS does actually provide this configuration. They have an automatic mode, just like Framework, but they also have configurable manual modes that allocate 1 GB, 2 GB, 4 GB, or 8 GB of memory. I am not sure if 64 GB RAM enables even more options in the BIOS. So it should be possible to implement this feature.
Using Smokeless_UMAF you should be able to access the UMA buffer size on any modern AMD system even if its not exposed in the BIOS.
Boot into a FAT32 drive with the downloaded Zip extracted to the root GitHub - DavidS95/Smokeless_UMAF, and then use the “Device Manager” section to locate the UMA buffer size.
The location within the utility is at
Device Manager > AMD CBS > NBIO Common Options.
Once set, back out and apply the changes, rebooting your system. You can then confirm that the option has been applied by checking your system monitor of choice (Windows > Task Manager, Linux > Whatever) and checking to see if your available RAM has decreased (indicating greater allocation of memory to the iGPU).
This may not remain applied through BIOS updates, and may cause hardware damage if misconfigured, and may result in an unbootable system if misused or by chance. You may need to reset your BIOS if that’s the case, if possible by your laptop manufacturer.
Have you had a chance to try this out on a Framework? When I tried last, there was no way to set a value beyond the ones already surfaced in the official BIOS.
Btw, I had a somewhat similar experience on a ThinkPad P14s too. There, additional values for the UMA size were visible in Smokeless, but wouldn’t be accepted when I tried to apply them.
UMA specified and then selected 16G, I saw on another post that it resets back, but I think it only resets back if you enter the bios again. From smokeless just select you booting device and proceed as normal
Wish I could assign more RAM to the GPU but going from 4 to 16 does make a huge difference in legacy or experimental software that does not fully support dynamic allocation. LM studio (im lazy) works faster for larger models. Will try llama on the cli later maybe even some SD. SD will still will be slow but at least it wont crash and we dont need to run on low ram mode.
Update: just tried crashing, not hard to accomplish. the vram goes back to normal if you sudden hard reboot. will try normal reboot and see what happens
One final update: The variable goes back to auto after each reboot. I wonder if i can make a small partition to keep smokeless there instead of using a USB drive each time. Im also wondering if there is a way to add this as a boot script of sorts… I would need to see how smokeless does its thing before.
