I think your idea would end up making karts indefinitely better. Keep in mind that they designed this game (specifically the new tracks), so that certain body types are better suited to different tracks. Inside drifting is already best on some tracks, as is outside drifting. Giving karts mushroom level turbo would probably tip the balance too much, even without the ability to go off road.
For example, I have a Rosalina kart combo that has near maxed out speed stat, as well as good handling, so if you gave me a mushroom level boost for my drifts, I would absolutely dominate some tracks, for example, GBA Mario Circuit, since the roads are so wide. Even more so if I use Fire Hopping.