You need to get the program "Parameters.exe" to do this. If you don't already have it, I can provide you with a download for it pretty soon.
Once you have that program, you need to go to Smash Explorer, and go to data(your region)>param>ui, and extract "ui_character_db.bin".
You then open that in Parameters (just go to File>Open), and then you will see this:
Then you collapse that, and it looks like this:
This is pretty daunting at first, but basically, each entry represents a character (including non-playable ones like Master Hand).
In order to find a character's entry, you need to know what their hexidecimal ID is, and convert it to a decimal. There is a list for this I can also send you (I can also convert every character entry to decimal too for you, it's actually something I've been meaning to do myself).
Since you're looking for Ryu though, I can just give you it up front - his hexidecimal value is 0x34, and converted to decimal, that's 52.
You'd think that means he is Entry 52, but that's actually not the case, but he is in around that area. So go to entry 52 by double clicking it, and you'll see:
To find which character this is, you look at "Name 6". Name 6's Value is 51 in this case, which is where Mewtwo is. So just go one down to Entry 53, and that's where Ryu is.
Now to actually make it so that Ryu's 5th costume uses your Evil Ryu voice mod and only that costume, go down to view Name 21 through 36.
21 through 36 essentially dictate what voice file the character uses. Since Ryu only has 8 costumes by default, 29 through 36 are empty with a "0". 21 through 28 are "1". Name 25 is Ryu's 5th costume, so change it's value to "2".
Now save that file, and drag it back into Smash Explorer, in both data(your region)>param>ui, and data>param>ui.
As one final step, go to your nus3bank file for your voice mod. You likely have it named "snd_vc_Ryu". Change it's name to "snd_vc_Ryu_c01". And add that to Ryu's sound folder within the data section and the data(your region) section.
Now just build the mod and it should work, only Ryu's fifth costume should have your voice mod.
As a bonus tip, Parameters is also where you can add additional costume slots. To do that, when your are viewing a character's entry, look for Name 7. Every playable character except Little Mac has this value set to "8" by default. You simply change it to the desired amount, up to 16 (it's possible to go above 16 but this requires extra steps). So in this case, you likely have an Evil Ryu skin to go along with your Evil Ryu voice mod. So change Ryu's Name 7 Value to 9.
Then go to the folder for the costume mod you downloaded. If it's for the 5th costume slot, it will be named "c04". Change that to "c08", and drag it into the folder where Ryu's costumes are in Smash Explorer. And you can build the mod and it will take effect.
Also, when you make changes to a parameter file in Parameters, remember to hit File>Save and overwrite the one you extracted. The program does not autosave upon exiting it.
