All of the Persona games have a minimal amount of story overlap (excepting maybe P4:Arena), so in that regard you can start wherever you want. 3 and 4 are both fairly good places to start, actually. Persona 3 Portable also has improvements from the battle system in 4 that were not in the original PS2 release; and since you have a Vita, I would readily recommend it.
1 and 2 are very different beasts, and I would not really recommend them as starting places unless you're already familiar with other MegaTen titles.