Just to talk about the reason old character like Pit and Ice climbers was added in Smash back in the days, but today people don't see old characters like this happening in the fighter pass is simply because Sakurai didn't directly choose the character for the fighter pass. Characters like Ice climbers, Pit, R.O.B. and arguably, Ness for example are what I always call "Sakurai choices". They're basically characters that where choose because Sakurai love their franchises and have a lot of nostalgia through these games. I think that these type of characters might come back on the spotlight if we do have a Smash 6 and if Sakurai came back, but that's really unsure.
First parties in the pass in general feels very unlikely. Smash Ultimate is far from a port as some would have insisted, but it borrows heavily from it (the physics are generally similar, graphically similar (Ult has better lighting and texture but that's mostly it)) and the smaller base roster only adds to this. If the game'd been given a few more years (2020 release?) then we prolly would have gotten a much larger base roster, including a "retro" character(s) and whatnot. I'm glad Ultimate is out, but I can really only imagine how different things would have been if it'd been given 2 more years to develop and grow in roster size. As a retro fighter, I'd love Ayumi or Takamaru- both were cut from Melee because their source games weren't known in America and were thereby even moreso "who" than the Icies.
Outside of personal opinion (Sakurai does base his choices off of popularity to a degree but that don't mean Walu would have gotten in) and the obvious Rex, ARMs, etc, who do y'all think would have gotten in as base roster if the roster was decided in later 2017/2018 and game released in 2020?
Or maybe that trailer was just not canon in the Mario lore (if the Mario universe even have a lore). I mean honestly I agree that Doomguy have a lot going on for him, but saying that this is the biggest reason is really a stretch. It's not like Nintendo will just stop using Luigi, one of their most popular character from their biggest franchises, just because of a trailer.
Mario canon is basically nonexistant. The universe's reset at least once in its lore, so who knows what was made invalid by that reset. And then you've got spinoffs of varying degrees, ranging from games where you can only barely recall this is Mario canon (Warioware, Donkey Kong) to games where you go-kart the capital G God(ess) of their world and the dictator of a foreign nation.
It's probably much too large a series to properly have canon anyways. Look at Pokemon and Zelda- there's multiple timelines and general mass confusion, when people try to organise it (Hyrule Historia did help Zelda some but BOTW is apparently in every timeline, and there's no answer for that). And then you got stuff like Kid Icarus, where a mainline game (OMaM) isn't canon but it's implied Smash is canon to it???
Nintendo generally only has more solid canon for the more serious (Metroid, SF) or more small (ARMs, Splatoon, etc) games, from what I can tell