While I get and agree with part of what you are saying (like most men respecting consent), there are thinks I disagree. As it happens, some men who don't respect consent are friends with men who do respect consent and use some stereotypes and "social agreements" as a way to bypass consent. For example "women are sluts" is a way to say that if a woman wear certain clothes, then that is equal to consent where no consent has been given. In that situation, men who do respect consent might believe the friend who doesn't respect consent when he lies and claim consent was given and defend him as "wrongly accused". The key here is that the rapist is deceiving both women and male friends.

Since rapist looks exactly the same than normal respectful men and act exactly the same until they earn enough trust to be able to rape, some women act as if all men were "potential rapist" and the non-rapist feel offended because they can't understand why they are being treated that way. And that in turn make the rapist more believable to them when he claims that the women are accusing them wrongly.

By the way, rapist can pretend to be "woke" or "non uber masculine" too. In fact, rapist would pretend to be decent and surround themselves by decent people since that is the best disguise/way to approach victims. To use an known example: pedophiles try to earn access to child by getting positions that would grant them both access to children and the trust of parents. That is why we are finding so many coaches and priest as rapist and pedophiles.

With women or men rapist work in a similar way. The rapist would earn the trust of a group of nice people to get access to victims and for victims to lower their guard. You might surround yourself from the best people and still be unlucky and find a rapist that is good at pretending he is a good person. They know that what they do is wrong, so they deceive people around them and some of them become very good at deceiving. That is why listening to survivors is important. Sometimes is the only way to identify rapist and predators. And I don't think "blindly believe them", I mean listening to both and locking for contradictions on what is said. Many times rapist contradict themselves or change their versions, yet men believe them anyway while trying to disprove at any cost what the survivor says. Or try to justify what the rapist did as a "misunderstanding" because thinking that they might have been used to get close to the victim hurts too much. But the fact is that in those situation men have been betrayed too. By the way, rapist use women in this way too.

But I get what you are saying, most men are respectful and not rapist. Those men are being betrayed by the rapist too and the rapist take advantage of their loyalty (the "bros over hoes" thing is that in a nutshell). So what we can ask of the men in our lives is to pay attention to the other men who might not be as trustworthy as them and cut the non-trustworthy from their lives. Although I suspect that most men already do that as best as they can and the problem of rapist being good deceivers remains.