I think that the problem is not age gap itself. Some relationships or flings with a big age gap work well because there is a balance of power and the needs and wants are aligned.
But young women also experience the "predators", which are older men who would try to get them drunk and pay for stuff in order for them to feel indebted and "won't take no for an answer" and would ignore any boundary under the pretense that their "experience" makes them "know what you really want". And many of those predators want unequal relationships where they hold the power. They are common enough to be a "type". Those are the ones who give a "bad rep".
For the record, I don't think that Leo DiCaprio is among those. Since he is famous, women do know what to expect from him: one or two years of fun with someone who is handsome. I don't think that most actresses and models who date him delude themselves thinking that it would last. And I don't think he promisses that either. And the jokes he gets for that are just that, jokes. It works because the core of the joke is that "he can make it work when others won't" (Let's be honest here, most 50 something aren't DiCaprios in the same way that most 50 something aren't Angelinas XD).
But maybe we could get away with the stigma if we start pointing out the predatory behaviour, so we no longer asociate it to age-gap, but just to predators. And I think most women have experienced at least one of those predators in life. I know I have and the first time I was a teenager and he wanted to get me drunk, go figure.