You can add vim files to .gitignore file because vim files end with ‘~’
It can also create swap files that have the swp and swo extensions. To remove those you can use the lines:
*.swp
*.swo
Which helps you to ignore all temporary vim files from a project.
For ignoring globally, you can create a .gitignore file in your home and use the following command:
git config --global core.excludesfile ~/.gitignore
Then you just need to add the files that you want to ignore. Thus, git ignore your temporary vim files.