The problem is still the fact it does not act as a handheld
Sure you can pick it up and play anywhere if you have space in your bag, but not only is it harder to move around but it also has less battery life, which are contrary to what people look for in a handheld. You're better off playing it somewhere where you can plug it in and charge it up.
Then there's the hardware which is the one of a home console, not a portable. You can't really justify making a 2D Zelda game, whose production values are lower than 3D ones, in a console who can support BotW and selling them for the same price, we all know how well Four Swords Adventure went. With that, there's also the lower costs for the consumer in the handheld, not only does the system cost less, its games do too, making it nore accessible for those without the money to constantly buy console games(3 handheld games are equivalent to 2 console ones, by US standard prices).
And finally, simply being practical. Which is easier: turning on your Wii U or 3DS for a quick match of Smash when you have some free time?