It is obvious that it is not about sex, since plenty of men go to the beach where women only wear bikinis (and in europe they go topless) and nothing happens.
But if I have to make a guess, I would say that it works the same way that bullying. Somewhere I read that the main bully use a “henchman” or “buddy” to do the actual harashment. So the henchman would be the one that insults, while the actual bully might play the part of “friend of the victim”. And then, there is the crowd or spectators, who see the bullying and take part in the sense that they would laught about the “jokes” and harashment that the henchman do or remain quiet in order to not attract the attention of the bully.
Usually, the bully would talk bad about the victim to the henchman or the crowd, but would sacrifice the henchman if things get ugly and someone is close to discover the bullying.
If you apply the same logic to sexual harassment, then most of that men who strongly deny it is indeed bullying and abuse as oposed to “natural sexual urges”, then you’ll get that some of them belong to the crowd, while others are the main bullies.
Basically, speak up would drawn the attention to them (And they would be bullied instead). It is easy to think that “it is not their problem”. For others it is convenient to “put women in their places”. They might not be doing the harassment themselves, but they agree with putting women in their places and, as a plus side, they could play the role of the “protector”. After all, if there are harassers, women would need protectors and “owe” them for that protection.
At the end of the day, admiting that it is bullying would make more difficult to look the other way. And acting have a cost. They no longer could considered themselves “good guys” if they have take part on the harassment and do nothing.
To sum up, sexual harassment is not limited to the one person who rape, touch you or subject you to the sexual comments, whatever. The other people play a part by denying it happened in the same place, victim blaming, etc. The rest are the “crowd” and they also are needed to gaslight the victim. Since kneejerk reaction of people is “notallmen”, it is no surprise that they don’t want to admit to be part of it, even if it is unknowingly.
It is like killing a pet by accident. You might not have wanted that result, but some people would deny having done that even if caught in the action.
