You can try increasing the GTT pool with something like:
/etc/modprobe.d/increase_amd_memory.conf
#Otherwise it's capped to only half the RAM
options amdgpu gttsize=90000 #in MB
options ttm pages_limit=22500000 #4k per page, 90GB total
options ttm page_pool_size=22500000