It's pretty simple. Most people won't do anything unless it directly impacts them.
This has been true time and time again throughout history as well. People only revolt or start an uprising when things get so bad that it directly influences them. Even then most won't do anything unless they aren't able to ignore it anymore.
I mainly tried lemmy.world, lemmy.ml, beehaw.org and a few others. I did try them with and without "all" and it was cumbersome regardless. Either I was served posts with zero or negative upvotes that to me do not classify as "hot" or my feed did not update at all.
What's weird to me is that kbin.social was the only one I tried that was not affect by any of this strangeness.