Why am I the bad guy?

Why is it when Caucasian women tell men of other races that they are not interested, that they are supposedly racist or still in middle school? But yet it is ok for men, of any race to say they don