For me, if I had to actually define some of these words, it would skew even more by gender, depending on what you mean by "define". For example, I know that a peony is a flower, but I wouldn't be able to draw you one, nor would I be able to identify it other than by process of elimination. Similarly I know that "taupe" is a color, but I don't really know which color (I mean, obviously these are now bad examples because I look thinks up when I don't know them, but the point is still relevant).
I'm not that surprised that many of the "female" words have to do with clothing that only women wear. Men don't tend to wear anything all that elaborate, and often it's unisex anyway. It's possible that that's an artifact of the fact that women tend to be more interested in fashion than men, though - I guess I don't really know the difference between Oxfords and boat shoes, so maybe the fact that there's no comparable set of fashion terms for men is because even men don't know the terms for the clothes they might wear.
I'm not that surprised that many of the "female" words have to do with clothing that only women wear. Men don't tend to wear anything all that elaborate, and often it's unisex anyway. It's possible that that's an artifact of the fact that women tend to be more interested in fashion than men, though - I guess I don't really know the difference between Oxfords and boat shoes, so maybe the fact that there's no comparable set of fashion terms for men is because even men don't know the terms for the clothes they might wear.