I want to use my Framework as an external monitor/keyboard/touchpad for a Raspberry Pi. Can you help think of some different ways of doing this?
The idea is to practice with setting up partitions, grub, multiple linux installs, etc, without risking my primary OS (Fedora) and files needed for school. I also want to avoid being stuck at a desk cluttered with peripherals.
Is there something I could do to plug-and-play a Raspberry Pi off my Framework? Is there something I could install on Fedora to facilitate this? Thank you for your thoughts!
I haven’t used it in years, so I have no idea how it’s changed since then, but I used to use an open source program called Synergy back in 2012-2016 to sync a single keyboard and mouse cursor between two computers, one running Windows and the other FreeBSD. It worked really well - I was always stunned at how seamless it was, it even did stuff like sync the clipboard and if one computer went to sleep, it’d make the other one too iirc.
Unfortunately, that doesn’t let you use the FW’s screen as a monitor though, the Raspberry Pi would need its own monitor to see with. In that case, you might want to use a screen sharing solution like VNC - it won’t be as seamless or smooth as synergy, especially over a slower network connection, but it will let you view the screen of your Pi in a window on your FW, so you can run your RPI run headless.
Lastly, if you have enough storage and your goal is to learn how to use Linux in a safe environment, you might want to check out a virtual machine program like VirtualBox. It lets you run any OS inside a sandboxed environment where it doesn’t have any access to your files or any ability to touch the host OS, so it might be perfect if you want to try something out before committing to using it on real hardware. I’ve used it myself on the FW and the FW is definitely powerful to run it at a decent speed.
All of these options are very helpful, thanks! Sometimes I need help simplifying what I have in mind. I guess virtualbox would work just as well for now.
If you want to run Raspberry Pi OS in a VM, the strictest way to do it is using QEMU. QEMU can present a different architecture (in this case ARM) to the virtual OS.
VirtualBox is much easier to use but it can only present the same architecture as the host (x86) to the virtual OS. You can’t install Raspberry Pi OS for ARM in VirtualBox.
Fortunately there’s Raspberry Pi Desktop which is made for x86 and should install perfectly in VirtualBox. Its interface should be identical to Raspberry Pi OS on an actual Raspberry Pi ARM board.
The functionality that OP is asking for would be amazing in a module. Or built into the next iteration of the framework laptop in general. I love this laptop but I will say that the GPD pocket 3 does have this as a leg up. Very different devices but at the same time it’s a dream to have a laptop that can double as a portable display keyboard and mouse not just for travel but also for utility purposes. I would love to see this function in the next framework device.
Okay, now this is neat. Bookmarked for looking into this over the weekend. Just a heads up - I have not dug into this yet, but assuming this is like similar tools it may be an over the network kind of thing. So latency may make itself known. But I will be interested in your experiences with it.
Not directly helpful but maybe a solution approach for future development of Framework modules: there exists a module for quite this task for the GPD Pocket 3
A dummy mainboard that is basically a dp/hdmi to edp converter for the screen and a usb converter for the touchpad keyboard + maybe power management if you want to use it with the battery.
The gpd thing as an expansion card, video input like the gpd could be done with a generic capture chip like you find on those 10$ chinesium usb capture cards. The keyboard/touchpad part looks a bit more complicated, you’d probably need software + an external microcontroller (or maybe you could bully the ec to do that) to act as a usb host to send keaboard mouse events to an external device. Could probably use a lot of stuff from pikvm for that.
I went to read up how it works and apparently there’s some hardware involved for such a feature, so we can’t like just enable it in firmware. At least that’s how the GPD Pocket 3 does it. The expansion card is more like to provide the connectors.
For the pocket, I remember it is actually just a connecting module with HDMI and USB which makes connection to the mainboard which has the actual KVM module itself. So kinda given up until we see hardware support if it ever comes.
If it was just a capture card, not sure if it is any better than buying an off the shelf USB capture card though.
The linked gpd in the video exposes the video as a webcam, that’s a capture card. That part would be easy, just cram one of those generic hdmi to usb3 stricks into a module, the keyboard stuff is a bit more complicated but would also be doable, you might even be able to bully the ec to reroute the kb and touchpad somewhere else without hardware modifications.
If you wanted to actually expose the display externally you’d definitely need mainboard changes and I really don’t see that happening as it’s space and money expensive and really of limited value. Unless you want to do the dummy mainboard thingm then it would be allmost trivial (at least the displayport part, if you wanted hdmi it would be somewhat harder).
I was able to take some pictures: open device expansion port expansion port/mainboard from other side
You can see 3 modules - KVM, USB-A and serial port.
Both KVM and USB are using the same pins while serial port is using totally different pins. USB has near to no electronics on the PCB (just some very tiny chips, maybe redrivers?) while serial port has a microchip onboard.
KVM module is much bigger than the other two and has a lot of electronics which is covered under a metal screen on one side and black tape on the other. I was able to remove the black tape but there is not much interesting underneath. The interesting stuff is happening under the metal screen - which is soldered to the PCB and even has some protective (?) stuff on the solder points. I don’t think I’m capable of removing the metal screen without damage or even resoldering it after it. But it is visible that there a at least two chips on the PCB recognisable from the solder points on the visible side of the PCB. So maybe there is a HDMI-capture-chip and some kind of USB-host-chip or microcontroller which both connect via a USB-connection to the mainboard?
Next to the mainboard connector is a chip called “ITE IT5570VG-128” which iFixit found in Steam Deck too and seems to be some kind of embedded controller. But maybe totally unrelated to the expansion cards.
Please let me know if there is something more interesting to photograph or to blurry to be seen. I can take more pictures.