LLM Benchmark (AMD 7840u)

Framework Laptop 13 Linux
1

For anyone interested, this is my LLM benchmark on my Framework 13 equipped with the AMD 7840u mainboard (32gb RAM), running KDE Neon 6 (based on Ubuntu 22.04 LTS), kernel is Linux fw 6.5.0-1020-oem, cpu-governor on balanced. (laptop plugged on AC)


-------Linux----------

No GPU detected.
{
    "mistral:7b": "12.12",
    "gemma:2b": "26.39",
    "gemma:7b": "9.75",
    "llama2:7b": "12.01",
    "llama2:13b": "6.82",
    "llava:7b": "12.67",
    "llava:13b": "7.02",
    "uuid": "c830e285-c9af-5bcf-b477-85e89d03a7bf",
    "ollama_version": "0.1.32"
}
----------
====================
-------Linux----------

No GPU detected.
-------Linux----------

No GPU detected.
{
    "system": "Linux",
    "memory": 30.65096664428711,
    "cpu": "AMD Ryzen 7 7840U w/ Radeon  780M Graphics",
    "gpu": "Advanced Micro Devices, Inc. [AMD/ATI]",
    "os_version": "KDE neon 6.0",
    "system_name": "Linux",
    "uuid": "c830e285-c9af-5bcf-b477-85e89d03a7bf"
}

benchmark tool used : https://github.com/aidatatools/ollama-benchmark

2

Interesting benchmark. Here are my results with same laptop!

-------Linux----------

No GPU detected.
{
    "mistral:7b": "12.41",
    "gemma:2b": "28.17",
    "gemma:7b": "9.97",
    "llama2:7b": "12.54",
    "llama2:13b": "6.87",
    "llava:7b": "12.85",
    "llava:13b": "7.03",
    "uuid": "dff15406-d2b5-5c70-bed1-xxxxxxxxxxxx",
    "ollama_version": "0.1.32"
}
----------
====================
-------Linux----------

No GPU detected.
-------Linux----------

No GPU detected.
{
    "system": "Linux",
    "memory": 58.662559509277344,
    "cpu": "AMD Ryzen 7 7840U w/ Radeon  780M Graphics",
    "gpu": "no_gpu",
    "os_version": "Fedora Linux 39 (Sway)",
    "system_name": "Linux",
    "uuid": "dff15406-d2b5-5c70-bed1-xxxxxxxxxxxx"
}
1 Like
3

Anyone with 16 + dGPU?

4

Here’s the results with my 13" 1240p. Running nixos with 32GB of RAM.

{
    "mistral:7b": "5.53",
    "llama3:8b": "4.98",
    "phi3:3.8b": "8.17",
    "gemma:2b": "10.44",
    "gemma:7b": "4.16",
    "llava:7b": "5.29",
    "llava:13b": "3.24",
    "uuid": "d340a0a4-05cf-59b8-8bef-b37c560083d5",
    "ollama_version": "0.1.38"
}

And the system details:

{
    "system": "Linux",
    "memory": 31.06365203857422,
    "cpu": "12th Gen Intel(R) Core(TM) i5-1240P",
    "gpu": "unknown",
    "os_version": "\"NixOS 24.05 (Uakari)\"",
    "system_name": "Linux",
    "uuid": "d340a0a4-05cf-59b8-8bef-b37c560083d5"
}
5

I know this is in the 13 forum, but here you go, Framework 16. This was interesting:

-------Linux----------

No GPU detected.
{
    "mistral:7b": "41.25",
    "llama3:8b": "39.19",
    "phi3:3.8b": "51.70",
    "gemma:2b": "92.49",
    "gemma:7b": "36.76",
    "llava:7b": "43.84",
    "llava:13b": "4.35",
    "uuid": "bccd6f86-8b2d-5410-92fc-7120dbdd81f9",
    "ollama_version": "0.1.38"
}
----------
====================
-------Linux----------

No GPU detected.
-------Linux----------

No GPU detected.
{
    "system": "Linux",
    "memory": 58.668739318847656,
    "cpu": "AMD Ryzen 9 7940HS w/ Radeon 780M Graphics",
    "gpu": "Phoenix1",
    "os_version": "Arch Linux",
    "system_name": "Linux",
    "uuid": "bccd6f86-8b2d-5410-92fc-7120dbdd81f9"
}
----------
1 Like
6

I haven’t noticed this is the 13 forum. In any case, thank you for the benchmark

7

Here’s my performance on Arch Linux, 13" AMD:

-------Linux----------
No GPU detected.
Total memory size : 30.66 GB
cpu_info: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
gpu_info: no_gpu
os_version: "Arch Linux"
ollama_version: 0.1.41

Your machine UUID : a79f8302-d19d-5b65-a214-213535d44a16
-------Linux----------

No GPU detected.
{
    "mistral:7b": "6.09",
    "llama3:8b": "5.64",
    "phi3:3.8b": "9.54",
    "gemma:2b": "14.14",
    "gemma:7b": "4.85",
    "llava:7b": "6.21",
    "llava:13b": "3.43",
    "uuid": "a79f8302-d19d-5b65-a214-213535d44a16",
    "ollama_version": "0.1.41"
}

Any tips on getting some more performance out of it? I’m using the open source amdgpu drivers and have tried both ollama’s own install and the pacman/Arch-packaged version.

8

You’re number are strange, I also used the open source amdgpu drivers and ollama’s own install, maybe check your kernel version ?