Rune Factory is a spin-off of HM/SoS, to the point of the original game being called "Rune Factory: A Fantasy Harvest Moon" (Rūn Fakutorī: Shin Bokujō Monogatari in Japanese). It is conceived of as "Harvest Moon, but with a sword" and is a fusion with dungeon-crawling JRPGs. The upcoming/just-released-in-Japan Rune Factory 5 is actually made by the main Story of Seasons dev team, with Story of Seasons: Pioneers of Olive Town having been outsourced.
Story of Seasons and Rune Factory both take place in a series of vaguely-connected AUs, in that characters will often reappear with the same core personalities, yet make no or little acknowledgement of their previous appearances. There are some exceptions to this (ex/ Rune Factory Frontier is a direct-ish sequel to Rune Factory), but that's how the series generally operates.
This also can lead to some very confusing parallel development with characters like Maria/Mary/Marie all having probably been one person at one point, but Maria and Mary are now two distinct people (with Marie being Mary's name post-Natsume/Marvelous split).
I mean, Stardew Valley is very transparently inspired by the original SNES Harvest Moon and Story of Seasons: Pioneers of Olive Town is very transparently influenced by Stardew Valley, so there's very little that is Stardew-exclusive apart from the broad concept of swords and the specific magical creatures that appear in Stardew.... Story of Seasons has more potential to draw from just due to being a long-running series with a lot of games with unique mechanics (ex/ some games have a camera minigame or a bug-catching game or a farmer's market selling minigame, etc etc)....
That said, I don't think they'd differ that much in terms of things that would actually make it into a moveset. The biggest difference I can think of is that the magical beings in the Story of Seasons franchise are generally more interventionist then in Stardew. Like, in Stardew, you kind of have to seek them out, while the SoS Harvest Goddess regularly interrupts gameplay to chat with you about how many vegetables you've recently harvested. So, I think the Goddess and/or her sprites would be more likely to be in a SoS Farmer kit then the odds of Junimos showing up for a SV Farmer.
I imagine spirits would just be a bunch of animals, some gods, and then some villagers. It kind of depends on how wide they make the scope. I don't know if they'd be focusing on PoOT as it's new or FoMT as it is the second-newest and also a remake of one of the most beloved games in the franchise featuring some of the most reoccuring characters in the franchise....
On that note, what Farmer designs would people like to see?
Here's a list of the Farmers for those unfamiliar.