There are many reasons to dislike Nvidia on Linux. Here is a little thing that bugs me all the time, the updates. Normally the system updates would be quick and fast, but with the proprietary drivers of Nvidia involved, it gets quiet slow process. And I am not even talking about any other problem I encounter, just about the updates.
As an Archlinux based system user (EndeavourOS to be precise), I get new Kernel updates all the time. That means every time a new Kernel version is installed, the Nvidia driver DKMS has to be installed too. And that is basically the slowest part. But that's not too bad, even though it's doing this twice for each Kernel I have once.
What's more infuriating is, if you also happen to use Flatpaks for a very few applications. I really don't have many Flatpaks at all. Yet, the Nvidia drivers are installed in 7 versions or what?! And they are full downloads, each 340 MB or more. This takes ages and is the only part that takes long to update Flatpak system. I always do flatpak remove --unused to make sure nothing useless is present. /RANT (EDIT: Just typos corrected.)
The latest drivers on mint, 535, cause flickering on my monitors. There are a bunch of posts about this; when I installed them when they came out my screens went black and never recovered, had to power off manually, and then the top part of my monitors would just flicker every 15-30 seconds. I rolled back to 525, and now that it had been a couple months I had just tried to upgrade again recently but the problem remains, black screen, reboot, flickering.
I used Mint briefly on my desktop PC, and the Nvidia driver was the one thing that gave me issues. The recommended one was too old for some of the things I wanted to do, but the most recent one at the time made everything unstable.
Now I use EndeavourOS, and Arch seems to handle that driver a lot better.
The screen flickering issue is also sadly not limited to Mint. I was on Fedora over the last few months, and am now on Arch and still get that issue.
Also doesn't seem to be restricted to the DE either, just the driver itself being shitty it appears.