Some men see it that way. They try to hold the non-safe men accountable. But there are other men. The ones who consider that "it is not their problem and women should prioritize giving them the benefit of the doubt over their safety", and the ones who are actually happy with this because that allows them to play "hero" (by acting once if it suits them) or lowering the bar so much that they have to make no effort to be considered "nice".
I mean, if the bar for "nice" is just not doing terrible things, they only need to "not grope women", "not hit them" and can even get away with treating them bad and still be considered nice. So if they get people to consider "normal" (not even bad) the ones who do those terrible things, they won't need to actually be mindful and caring to women to get the "nice" title.
But it would be unfair to ignore that many men have also speak up against this. My suggestion is "pay attention" to how men react to other men doing things, good and bad. And to the "dog whistles". Then avoid the ones who are ok with other men being asholes even if they don't look like asholes. But rewards the ones that try to do good.