It's not impossible, but it's not exactly accurate to the core gameplay. Torna was the side story, remember? Even then, they weren't fighting purely on their own. Technically you fought solo blades in the base game, but you didn't play as them. Possible or not, it's simply a poor showing of what the game is. Especially when drivers and blades are actually plot important. If Pyra and Mythra could have just done it all on their own, why didn't they?
And, yes, solo Rex would have not properly portrayed the dynamic either, but gameplay, drivers were the important ones. Even in the story, there are several points (at least four with two of them being almost entire chapters) where Pyra and Mythra aren't there, but Rex is always present. In the final boss, they don't actually say that much.