Using pynput with shift and control key press and release in python3 not reliable on framework. When I run it on a Legion laptop with Ubuntu 22.04, works perfectly to capture signals from press and release of right shift and right control keys. Does not care which application is in focus and the background python3 app works fine. Same python3 app on the framework with with Ubuntu 22.04 does not work when the terminal, LibreOffice, Text Editor and lots of apps are in focus. When Visual Studio is in focus, it works fine. Firefox address bar in focus also works. Anyone have ideas on how to make pynput python3 module work well on the framework?
Well did you compare both your library/program versions and also the configs?
I have never used pynput, but different versions can cause this.
Also, what does this mean:
Using pynput with shift and control key press and release in python3 not reliable on framework
Does this mean that those events sometimes do not get triggered at all? Or that they send the wrong keys? Or that the events are async? Or something else?
Also, is this script that you’re running started with sudo or root permissions?
I am moving my stuff from the Legion to the framework. Versions are the same.
Not reliable means that it works with Code Studio and in FireFox, but not in Libreoffice, Gedit, Terminal when those are in foreground. Whether it triggers or not depends on the app in focus. With code studio in focus, always works. With LibreOffice in focus, does not work. Interesting that with the terminal in focus that launches the program it does not work, but if you put Code Studio in focus, works all the time, and stops/starts working as focus changes from one app to another. No such behavior on the Legion Laptop with same python3 program.
The program records audio from microphone when right Shift and right Control are held down at the same time. When released, it feeds the recorded file to openai whisper for speech to text translation. Then it takes the text, puts it in the clibboard and feeds Cntrl-V to paste the text into the current window where the cursor is. Works great on the Legion as a speech to text function, but only way now to use it is to dictate into Code Studio, copy from there, and paste into to target app such as OpenOffice writer.
Not sure why certain apps allow it to work fine, while changing focus to another app make it stop working.
Are the kernel versions and software updates at the same state between laptops?
I have never used pynput either. But what you describe seems to me to be the behaviour of an application developed for XWindows used under Wayland. Could it be, that your desktop on Legion was running with pure XWindows, while you have Wayland on Framework?
On Wayland applications only see keyboard and mouse events when they have the focus. On XWindows applications always can see all events. On Wayland XWindows applications use the XWayland server which translates their XWindows API calls to Wayland. XWayland can be considered as a Wayland application, that gets the focus if any one of the XWindows applications gets it. The consequence is that pynput can only see keyboard events, while an XWindows application has the focus, but not if the mouse pointer is in the window of a Wayland application.
You can check, which type of system your desktop or your applications are using with the xinput tool. (If not yet installed sudo apt get xinput). On a wayland desktop calling xinput without any parameters will produce an output like this:
WARNING: running xinput against an Xwayland server. See the xinput man page for details.
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ xwayland-pointer:15 id=6 [slave pointer (2)]
⎜ ↳ xwayland-relative-pointer:15 id=7 [slave pointer (2)]
⎜ ↳ xwayland-pointer-gestures:15 id=8 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ xwayland-keyboard:15 id=9 [slave keyboard (3)]
Now look for the id of the xwayland-keyboard. In our case it’s 9. If you type something like xinput test 9 in a terminal, you will see output for each key press and release in any application using XWayland. And if my assumption is true, this will be the applications that work with pynput.
A workaround then would be to switch off Wayland completely or find other ways to run the applications in question on the XWayland server. But be aware of the security risk that arises from the fact that other applications can see sensitive information like passwords.
1 Like
Thank you. You are absolutely correct. It is exactly as you described. I do understand the security concerns. I am using this on my personal laptop so I can dictate into any of the windows without using cloud based services. Using a python script running in a separate window, I record my speech to a file when right ctrl and right shift are held down together. When released, the file is fed to openai whisper and returns text. The text is fed to the clipboard and a ctrl-V is simulated on the keyboard to enter the text into the foreground app. Worked great on the legion laptop I am transitioning from. Love the framework but losing this functionality is something that is very important to me. Any resources you could point me to so I can recreate this functionality with Wayland? If not, is there a way to make the framework give me full time Xwindows for all apps?
I want to thank you again for your help. Your expertise makes the situation much clearer and gets me closer to a solution.
Just did a little digging and I see that this is something that Wayland may be working on. They are potentially replacing pynput with uinput, but there are security issues not resolved yet. These include having to get escalated privileges to get specific keyboard tables and may only allow things to work as su at this time. Maybe available in Wayland in the future but does not seem likely.
To make my framework functional for my purposes since I am the only one using it, I will turn off Wayland and try it again.
Yup. That did it! Thank you for the help.
1 Like
Awesome, glad this is resolved.
I’m glad I could help and I’m even more pleased that you have found a solution.
1 Like