True. And sometime it is the way they justify their wages. What he did is gaslighting and it was probably framed as a way to increase control on the workers and "productivity". They usually choose the productive workers instead the others, since confronting the slackers would be hard and would bring few results, while pressuring the ones that work well could bring more work.
I've suffered something similar. The "explanation" (for your peace of mind) is that the more efficient you are, the more they want to squeeze out of you. That doesn't make it right or less violent, but it is the reason. For them, you are not a person, you are a "resource" and they want to reach their "KPIs" so they can get a bonus at the end of the year.
With remote work, many middle managers that used to go to pointless meetings to justify their work, now need to do this kind of thing to pretend they are doing something. And they target the workers they think won't raise problems: mothers, fathers, young women, people of color, people who need the job...
Ask yourself this, if he has to justify his work, what does he has to show? Probably nothing.
So you are right to feel angry, but also take this as a reminder of your own value. You must be doing a very damn good job. All this is a signal that you could be doing his job better than him and he feels threatened. And that gives you power. Once you realize that you are good at what you do, you can start looking for a better job (don't quit until you have another one lined up). In the CV start listing the task you are doing (not the name of the position or just a generic one with the task). Chances are that you are doing part of his job and are already qualified for a raise or a better position, but you won't get it at the same company (he would block you). When asked at interviews, don't talk about your boss, just say that you wanted to progress but at your current company that wasn't possible but that you are ready for more responsibilities (and ask for more money, obviously).
What he has done is not fair, but sadly, the best answer is to take care of yourself and get away from the bully. On the other hand, if rotation on his team is high (that is, if you leave and more people do), he would be punished by corporate because that is seen as bad managing (it is).
If you can, get away from him, you have no obligation to put up with a bully.