I like RPGs and games like Ace Attorney or Zero Escape. I prefer games with a strong plot and characters.
That's one of the reasons the Persona seems so enticing to me, it appears to be the whole package.
I appreciate games with unique style and game mechanics too. That's why Gravity Rush and Tearaway caught my attention.
If you like AA and Zero escape, Danganronpa is an absolute must. The second game comes out fall as well. I'm going to assume you have a 3DS, so you won't need to get Zero Escape for Vita then. So for VNs, definitely get DR. By Chunsoft and very Zero-Escape-esque. Avoid googling the game due to spoilers being everywhere.
RPGs, I'd suggest Trails in the Sky. The second one is still being localized. Good classic style JRPG with good story. Both games will be offered on Steam though, so I wouldn't say it's an exclusive. If you're interested in strategy games, Tactics Ogre and Final Fantasy Tactics are really good and strong in the story and character department.
You already mentioned the Persona games. Just remember that there's P2:IS and P2:EP. Innocent Sin takes place first, so make sure you play it in order. The first Persona game (Persona 1) is pretty dated nowadays, but you might find it fun to see how the series started. Atlus usually does Persona sales every year, especially during the summer where it becomes half off. So hold off on buying all of them if you want until it happens.
Some good PS1 RPGs I would recommend would be Vagrant Story and Xenogears. If you played Xenoblade, Xenogears is worth checking out. Really different in terms of gameplay and story, but it's by some of the same staff. Any of the Final Fantasy games are worth getting if you never tried them out. Pretty much 1-10 are available for the Vita.
Metal Gear Solid 1 and the HD Collection are good games to look at. 1 and 3 are some of my favorite games just from the story.
It's worth it even if you might not think the gameplay would suit your style.
If you're into something weird and quirky, I'd look at Locoroco, Patapon, and Katamari Damacy. All are weird charming games, but have little to no story.