There currently isn’t a way to control the fan speed :Fan speed / curve control, and battery charging thresholds - #2 by Kieran_Levin
If/when that’s implemented in the BIOS then that would certainly fix it. But it may need addressed in the default fan profile depending on how many people have this issue. At least one other person is experiencing it that I’ve seen: OpenBSD on the Framework Laptop