There are many reasons to dislike Nvidia on Linux. Here is a little thing that bugs me all the time, the updates. Normally the system updates would be quick and fast, but with the proprietary drivers of Nvidia involved, it gets quiet slow process. And I am not even talking about any other problem I encounter, just about the updates.
As an Archlinux based system user (EndeavourOS to be precise), I get new Kernel updates all the time. That means every time a new Kernel version is installed, the Nvidia driver DKMS has to be installed too. And that is basically the slowest part. But that's not too bad, even though it's doing this twice for each Kernel I have once.
What's more infuriating is, if you also happen to use Flatpaks for a very few applications. I really don't have many Flatpaks at all. Yet, the Nvidia drivers are installed in 7 versions or what?! And they are full downloads, each 340 MB or more. This takes ages and is the only part that takes long to update Flatpak system. I always do flatpak remove --unused to make sure nothing useless is present. /RANT (EDIT: Just typos corrected.)
You can manually remove all of the previous drivers and their GL32 counterparts, your flatpaks will still run as long as you have the newest drivers.
I'm not sure why they don't get caught by the --unused flag, but they are definitely not needed from the flatpaks I've tested with.