Ohhh, ok you're right. That's why Michael Jordan was so good at baseball. And that Babe Ruth guy, what a fat slob. He must have been the worst.
Hitting a baseball is a very, very specialized skill. It wouldn't be too far-fetched to say that hitting a MLB pitch consistently and make it to at least first is one of the hardest things to do in all of sports. Babe Ruth was able to set a home run record only because his muscles were physically fit for the job. Michael Jordan on the other hand had muscles that were fit for Basketball. Ruth was in fact physically capable, just in a different way, much like the reason why Jordan sucked at Baseball but was amazing in Basketball. How high you jump and how swiftly you can position yourself in reference to the opposing team's offense/defense does not transition well into a sport like baseball. No doubt they still require a specific kind of finesse, but the muscles that Babe Ruth has would be entirely unnecessary for Jordan when playing basketball.
I suppose you can simplify it like this: Jordan was actually not fit because he did not have the proper finesse for playing baseball. Ruth on the other hand had enough body strength to favor his swing, a perfectly fine body for this sport. Having a six-pack doesn't automatically make you the strongest man alive. Different sports require different muscles.
I'm not arguing that sports do not require dexterity in order to be professional, but to insinuate that physical fitness is irrelevant for such sports is just plain wrong.