When using another AMD laptop, the Lenovo Flex 5 14", I notice that when plugged in, due to it switching to the high performance turbo mode, the laptop gets quite hot pretty fast.
The turbo boost mode lets the laptop turbo quite aggressively until hitting either thermal or power limits.
If you are having issues, then try turning turbo boost off. My average temps without turbo boost enabled (on battery) are around 50C while once turbo mode is up it jumps up to the high 80s.
When the laptop is plugged in, the OS automatically changes to the high power profile (with turbo boost enabled) which is likely why the laptop may get hot quite quickly.
Another possibility is a faulty cooling system, so if you want to, you can file a support ticket to check with Framework whether your cooler is faulty. Framework Support is quite busy so it might take some time before you get a reply.