That’s normal Windows weirdness and not related to the Framework laptop or the RTC battery.
See https://www.howtogeek.com/323390/how-to-fix-windows-and-linux-showing-different-times-when-dual-booting/