As far as I know.
First whistle always reorder your pikmin with RYBWP and points the first in that line, sequential whistle will change the pointer to the next Pikmin.
If you use pikmins with any move but the line maintain the order, the next whistle will change the cursor. If they are not sorted and you whistle again, it will be like the example above.
I have to confirm this though.
I just spent what felt like a couple hours in training mode on my 3DS committed to understanding the whistle and
It seems to work like this:
Look at all living Pikmin and select the one that appears earliest in the order.
Rotate the order such that the selected Pikmin appears at the end.
For example, when the game starts the order is RYBWP so if you attack with the red (selecting the yellow) then whistle you will end up with the red again.
Then since you selected a red the order changes to YBWPR.
If you were to then throw the yellow off the stage and pluck a white, despite where the living Pikmin are, whistling would select the blue and set the order to WPRYB.