Follow these steps to get a clean V2 install:
1. First off, uninstall any previous game versions. Use an uninstaller like cccleaner (it is free, just click on the download links and don't get "tricked" into buying the support version) or Revo Uninstaller (also free), as the windows tools are... poor. Also, check for any remaining registry edits that need to be removed. cccleaner/Revo can do that for you.
2. If you use Windows 7/Vista, Microsoft has added so-called "compatibility files". Basically, anything that gets changed in the C:\Program Files directory will be put into the compatibility files. So, you'll need to delete those files as well. Either click on "compatibility files" in the windows explorer, or go to C:\Users\YOUR-NAME-HERE\AppData\Local\VirtualStore and search for the Paradox folders in there.
3. Manually delete all remaining Victoria 2 folders
4. Reinstall the game. Do not install the game to C:\Program Files (this will save you a lot of trouble with the compatibility files). Instead, install it to C:\games\Victoria2, or any other directory not in C:\Program Files.
5. If you use Steam, then it is possible to move the entire Steamapps folder to somewhere else outside of C:\Program Files:
https://support.steampowered.com/kb_...7710-tdlc-0426
6. Right-click on the V2 icon and select the options to disable desktop composition, and to run as the Windows Admin user.
7. If you have Win7/Vista, disabling UAC can also help
8. That's it, run the game to see if it works. You might want to delete the contents of the \map\cache\ folder (in your V2 game directory). Do not delete the whole map folder!