Edit: A lot of people say, that GWM needs a melee weapon attack, but they miss Jesses point: While GWM requires a melee attack with a heavy weapon, Sharpshooters only criteria is an attack with a ranged weapon (not a ranged weapon attack). Jesse bases his claim on the fact, that a crossbow is still a ranged weapon, even if used as an improvised weapon for melee combat. That’s why it deals 1d4(!)+20 damage. (It works with any ranged, heavy weapon btw., so Longbow qualifies too.) Of course Jesse is playing the devils advocate here and of course, no somewhat sane Walter will allow this in any campaign ever, as it’s obviously not the intention behind these feats. But you could read it that way and that’s Jesses (paperthin) point. Besides: he finds the image of a barbarian running around recklessly smashing a crossbow over everyone’s head to just be hilarious.
The DM omnirule applies: If it breaks the game, then no, just no. But if it merely makes the game more ridiculous without giving the DM a splitting headache or driving players away from the table, allow it.
I suspect something like this would trigger the headache clause, and if I were DM I'd probably ask the player to pick which one he would rather it apply to, but not both.
-10 to hit will mean this person is doing an average of 6 damage an attack across an arbitrary number of attacks. I think you'd be fine.