[TRACKING] Touchpad interrupts & battery usage issues (idma64.2)

Hey thanks for the feedback! I agree this seems to be from processing the interrupts rather than the touchpad itself.

One problem I had was to keep motion consistent between tests this is why I went for resting a finger on the touchpad and peak power draw from moving. I think you are right however measuring total consumption rather than rate would be better.

To clarify “just touching” is me resting a finger on the touchpad and the additional power draws mentioned above are constant rather than just spikes.

I did some further testing only in Windows to remove Linux variables.

When using the touchpad vs an optical USB mouse the touchpad is on average using 0.9 watts more doing the same task.

I considered doing a total consumption comparison but the duration needed to get some reasonable numbers would be longer than is physically comfortable.

Makes sense, since the proccessor needs to do more computer to get where ur moving your finger and all that.

1 Like

Thanks that does make sense knowing that. So only a more efficient CPU will help. Seems I will be wanting that Ryzen upgrade ASAP!

1 Like

Just an observation …
I disabled 3 cpu cores in the bios and noticed that as a result powertop now only shows roughly 25% of the wake up counts when using the touchpad.

Fedora 35, kernel 5.18.11, 1185G7

1 Like

Hi, I’m Matt (the Linux Support Lead) and I wanted to see if I can replicate this. Now for the oddball questions that may not seem relevant.

Is this on a 11th or 12th gen Framework?

2 Likes

I’ve been experiencing this on a 11th gen Framework, and looking from the posts above they also seem to have 11th gen Framework laptops.

I was curious to see if this was an issue with 12th gen as well. On Arch running Linux 6.0.7-arch1-1, I get the following on my system between doing nothing and holding my finger on the touchpad measuring with turbostat:

# turbostat -c package -S --show IRQ,PkgWatt -i 1

IRQ		PkgWatt
227		0.53
203		0.52
216		0.53
238		0.52
261		0.77
1979	1.44
2295	1.58
2686	1.67
2463	1.58
3634	1.82
3577	1.78
3239	1.71
2311	1.51
3559	1.79
3528	1.79
3576	1.81
3592	1.84
2348	1.60
3543	1.80
3232	1.70
2186	1.47
3595	1.79
3575	1.79
3632	1.81
2593	1.53
3521	1.80
3563	1.78
3789	1.80
2649	1.56
2616	1.61
3562	1.79
3157	1.69
2630	1.62
3567	1.77
3649	1.77
1727	1.03
237		0.63
223		0.52
225		0.53
245		0.56

And here’s the comparable dstat numbers for reference (turbostat gives accurate package power usage so more useful for looking at real world effect?)

# dstat -t --top-int
----system---- ---most-frequent----
     time     |     interrupt      
10-11 14:51:38|i915              7 
10-11 14:51:39|i915             17 
10-11 14:51:40|i915             14 
10-11 14:51:41|i915             13 
10-11 14:51:42|idma64.2, i2c_designware.2 1107 
10-11 14:51:43|idma64.2, i2c_designware.2 2898 
10-11 14:51:44|idma64.2, i2c_designware.2 2858 
10-11 14:51:45|idma64.2, i2c_designware.2 2646 
10-11 14:51:46|idma64.2, i2c_designware.2 2926 
10-11 14:51:47|idma64.2, i2c_designware.2 2870 
10-11 14:51:48|idma64.2, i2c_designware.2 2715 
10-11 14:51:49|idma64.2, i2c_designware.2 1784 
10-11 14:51:50|idma64.2, i2c_designware.2 1881 
10-11 14:51:51|idma64.2, i2c_designware.2 1722 
10-11 14:51:52|idma64.2, i2c_designware.2 1493 
10-11 14:51:53|idma64.2, i2c_designware.2 2913 
10-11 14:51:54|idma64.2, i2c_designware.2 2897 
10-11 14:51:55|idma64.2, i2c_designware.2 2629 
10-11 14:51:56|idma64.2, i2c_designware.2 2777 
10-11 14:51:57|idma64.2, i2c_designware.2 2817 
10-11 14:51:58|idma64.2, i2c_designware.2 1688 
10-11 14:51:59|idma64.2, i2c_designware.2 2458 
10-11 14:52:00|idma64.2, i2c_designware.2  86 
10-11 14:52:01|i915             13 
10-11 14:52:02|nvme0q8          15 
10-11 14:52:03|i915             13

So, maybe someone with a Ryzen 6000 laptop can chime, but with my curiousity piqued, I also decided to test with my old 4800H PF5NU1G (running the same 6.0.7-arch1-1 kernel):

# turbostat -c package -S --show IRQ,PkgWatt,C3% -i 1

IRQ		C3%	PkgWatt
791		99.53	2.68
815		99.56	2.66
698		99.57	2.69
738		99.54	2.68
3794	88.70	3.40
5542	83.45	3.79
5534	76.96	3.76
5542	79.23	3.85
5507	85.05	3.79
5569	85.37	3.77
5518	83.33	3.74
5425	83.98	3.73
5435	65.37	3.74
5534	80.22	3.77
5555	85.38	3.74
5463	78.40	3.74
1178	87.22	3.42
768		99.54	2.66
668		99.59	2.67
664		99.56	2.66
804		99.41	2.68

As you can see, this seems to have about the same impact (more IRQs and worse on C-state residency even) on a Ryzen 4000 laptop.

@lhl I did similar on my 11th gen and got this:

IRQ	PkgWatt
649	0.90
613	0.92
2254	1.78
553	1.11
1939	1.59
604	0.91
627	0.98
604	0.94
579	0.92
548	0.98
593	0.99
570	0.94
592	0.93
643	1.06
590	0.98
497	0.97
555	0.86
616	0.98
355	0.84
606	1.02
563	0.94
605	1.01
608	0.89
627	0.92
572	0.90
539	0.92
579	0.94
510	0.88
576	0.95
579	0.93
1773	1.40
2510	1.93
491	0.79
1935	1.55
665	0.98
553	0.92
531	0.88
575	0.90
559	0.92
544	0.81
594	0.93
579	0.96
530	0.92
546	0.98
655	0.97
494	0.88
586	0.91
600	1.04
472	0.87
635	0.97
359	0.87
485	0.94
950	1.13
564	0.94
630	0.92
633	0.96
557	0.86
584	0.91
704	0.99
658	0.97
550	0.83
650	0.96
618	0.92
1018	1.05
640	0.97
490	0.91
386	0.86
646	1.02
578	0.90
587	0.95
630	0.95
566	0.95
563	0.90
658	1.02
658	1.04
565	0.91
572	0.94
665	0.96
574	0.94
544	1.62
345	0.83
645	0.98
733	1.05

This is on Fedora 37 (beta) with 6.0.7-*

I’ll need to spend some time with dstat Monday.

1 Like

I noticed the increase in CPU pakage power is substantially different to the increase in total system draw however the psys power domain (which also adds the PCH and eDRAM consumption AFAIK) is much more reflective of the change. Perhaps this is more about the PCH than CPU package?

My results from i5 11th gen Fedora 36 kernel 6.07

Initially the system was drawing 3.0W resting a finger increased this to 4.5W with only a ~0.5W increase to the pkg power.

IRQ	PkgWatt
548	0.49
511	0.62
463	0.61
479	0.49
552	0.67
573	0.58
459	0.48
515	0.62
422	0.53
565	0.57
800	0.73
2272	0.99
2184	0.97
2500	1.07
2362	1.05
2341	0.98
2618	1.14
2426	1.02
2262	1.00
2502	1.08
2392	0.99
2558	1.05
2436	1.05

Now “drawing” circles on the touchpad again idle at 3.0W rising to ~6.75W with 1.5W increase to pkg pwr.

IRQ	PkgWatt
442	0.45
481	0.45
485	0.55
495	0.49
441	0.55
423	0.47
453	0.46
434	0.49
419	0.53
382	0.45
423	0.51
2226	1.03
5549	1.90
5816	2.02
5573	2.00
5412	2.01
5510	2.02
5330	2.00
4921	1.96
5487	2.05
4798	1.95
5449	1.98
5859	2.03
5625	2.05
5583	2.04

Not a framework laptop, but Googling led me here as I am having this issue in Arch on a Dell G5 with an i7-10750H cpu. Same symptoms as above, verified with turbostat. ~1.25w idle and ~2w just moving the mouse. ~1.5w just touching the touchpad…

Edit: Can confirm that I get better battery life and lower powertop system usage on both Fedora and NixOS compared to Arch on this specific laptop… So unfortunately I’ll have to use something other than Arch on this laptop for now :frowning:

in Arch ~13-15 idle - 16-20w browsing the web (even a live ISO)
in Fedora or NixOS ~7.5w idle and 10-13w browsing the web so the difference is significant.

Edit2: installed NixOS including nvidia drivers… crummy battery life persisted, and even though in Arch I had enabled udev rules to disable the nvidia card as per arch wiki, my nvidia card still wasn’t disabling. Turns out at least on NixOS I was able to get the dGPU suspend working using the nix config posted in How to use Nvidia PRIME offload to run the X server on the integrated board - #15 by Denommus - Help - NixOS Discourse

Did anybody create a diff of the TLP configuration?

I think i have part of the reason why this is happening. I looked at this using fedora 37, and it seems the gpio interrupt is pinned to one core, and the i2c interrupts are pinned to a separate core. This means every time the touchpad fires an interrupt, both a high performance and efficiency core cluster have to wake up to service the touchpad! And these are probably not even in the same cluster, so lots of cache evictions etc might be happening as both core clusters power on and off and caches are cleared and filled.

What core type are TP interrupts handled by?

The second idea was to investigate what core the interrupts are handled by:

First you can look at the Cores on the system to map them to E or P cores. The E cores will have lower frequencies when looked at using:

lscpu --all --extended

The second thing was to find out what cores the interrupts were handled by:

cat /proc/interrupts

watch -n 1 "cat /proc/interrupts | grep designware"  
watch -n 1 "cat /proc/interrupts | grep PIXA"
            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15      CPU16      CPU17      CPU18      CPU19      
  43:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   43694516          0          0          0          0  IR-IO-APIC   43-fasteoi   i2c_designware.2, idma64.2
 135:          0          0    1711105          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0  intel-gpio    3  PIXA3854:00

And move the touchpad, you will see which core the touchpad interrupts are increasing on both.
However the PIXA GPIO interrupt is pinned to a high performance core, but the i2c-designware interrupt is pinned to an efficiency core.

Lets pin them both to the same core CPU2:

# echo 00004 > /proc/irq/43/smp_affinity
# echo 00004 > /proc/irq/135/smp_affinity # DOESNT WORK, IO error?

Power consumption when using the touchpad seems to drop from about 7W to 5W! (Baseline is 4W)

I would like to pin the PIXA intel-gpio interrupt to an efficiency core, but I have not figured out how to do this.

9 Likes

Wow, that’s an awesome find!

I replicate this on EndeavourOS (Arch), Zorin (Ubuntu), and a Fedora live USB environment. My dstat readouts closely resemble those posted by others above. EndeavourOS is somewhat worse, and Fedore is somewhat less-terrible. On Endeavour, a USB mouse throws about half as many interrupts as the trackpad; I didn’t think to test this on the other OS’s.

I seem to be seeing the same thing - the PIXA interrupt is going HAM on my first (P) core.

Which identifier does one pass to SMP Affinity from the following? I understand that the e cores are the second block, however what is the identifier? Its currently set to 00c0 for btoh, and the smp_affinity_list set to 6-7

I tried to investigate this. I really don’t know some things written in this thread. I tried to find a clue…

1 Like

For anyone testing this, here’s a minimally nicer command:

watch -n 1 "cat /proc/interrupts | grep 'CPU\|designware\|PIXA'"
1 Like

Here’s a simple systemd service touchpad-smp-affinity.service that pins the affinity of the designware interrupt to CPU2:

[Unit]
Description=Pin touchpad interrupts to CPU2
Documentation=https://community.frame.work/t/tracking-touchpad-interrupts-battery-usage-issues-idma64-2/13630

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/sh -c 'echo 2-2 > /proc/irq/$(grep designware.2 /proc/interrupts | cut -d ":" -f1 | xargs)/smp_affinity_list'
ExecStop=/usr/bin/sh -c 'echo "0-$(nproc --all --ignore=1)" > /proc/irq/$(grep designware.2 /proc/interrupts | cut -d ":" -f1 | xargs)/smp_affinity_list'

[Install]
WantedBy=basic.target

Test this by running

watch -n 1 "grep 'CPU\|designware\|PIXA' /proc/interrupts"

and checking if the numbers increase in the same column if using the touchpad.

I don’t think this is the final solution. As I understand from the thread, the number of generated interrupts is still too high. But this workaround is apparently better than nothing.

the large number of interrupts generated are probably from the i2c controller on the SOC, as each physical interrupt requires the CPU to transfer multiple bytes. A more optimized Linux kernel driver may be able to fix this.

Is there anyone here who’s willing to report this to the kernel folks?

4 Likes

Thanks everybody for the large amount of research that has gone into this.

From reading I understand that pinning interrupts to a certain core is a workaround that some people test with, but the process raises a few questions for me. I hope someone can shed a light here

  1. Why only pinning the designware.2 interrupts? On my 11th Gen I see designware.{0,1,2} interrupts listed in /proc/interrupts.
    What are the other 2 for?

  2. Why pinning to CPU 2?
    When I execute lscpu --all --extended multiple times over a range of a few minutes I see different cores with a high frequency - including CPU2. According to @Kieran_Levin the high frequency is how to recognize a P core, but if the E vs P core is not constant, the pinning ends up to be a hit & miss, no?
    Is there a way to dedicate a CPU as E/P when on battery?

  3. Why not pinning the PIXA interrupts too? Just to be explicit that both interrupts are actually handled by the same CPU.