Humans have sexual dimorphism, but it's a cultural thing that women wear skirts and men drink themselves to death instead of talking about their problems (both of these are jokes btw. I have a friend who wore kilts quite often and my mother drank herself to death)

Also, genetics is tricky, there a plenty of examples of people who do not fall into one category or another for these sexually dimorphic traits. There are people who have genetics from both sex, as well and differences in hormones distributions will causes these traits to appear or not appear.

Is a huge grey area.