Many people who aren’t religious have religiou families and have grow up surrounded by religious people. Some of those people have also studied religious text and think thougly about them. They just reached different conclusions. For example: catholics claim to be austere, the Vatican is very rich. They even have a bank and are shareholders of companies who sell weapons. Some people see a contradiction there and that drives them away from the catholic church.
But their countries keep giving money to the catholic church from their taxes, while they atheist get nothing. Just look at the guy who thinks that atheist should work more days for the same wage than a catholic that gets that day free at no cost. That is privilege. Even the fact that you don’t need to justify why do you take that precise day off compared with someone who want to take the 8 of march free is telling. A catholic can just say “it’s christmas” and that would be all. A feminist can’t just say “it’s women’s day” and be sure that she would be granted the day, even if it counts against her PTO. That is religious privilege since atheist doesn’t get any days where they are expected to take the day, neither are granted for free.