The reason that one version of the "go off" phrase (identical in every way to the other except for one word that specifies gender) might feel to you like it has different connotations is because we live in a patriarchal society that doesn't assign value the same across all genders. That's not an excuse to use the version of that phrase which misgenders someone.

And your example is really weird and obscures what's actually at issue. The difference in meaning between the words "bros" and "sissies" goes way beyond just a difference in gender. One is a common and generally affectionate term that men call each other when being friendly. The other is most often used as misogynistic term to insult men by disparaging their masculinity.