No. Because it is not the same physical action.

One is touching your shoulder with consent, other is touching your should without consent.

To give you an example. Lets say that a man finds a woman attractive and allows her to touch his penis. Then a man who is bigger but he doesn't find attractive goes and touch his penis without his consent.

Acording you it is the "same physical action" (touching a penis) and it is irrelevant it there was consent, right? And you can substitute that for the ass and any sexual act.

There is no double standard. There is consent or no consent. The thing is that some men feel that if a woman allows one men to touch her, she should allow the other men the same. Taken to the extreme, they seem to think that if a woman is a slut (enjoys sex with several men), she should agree to sex with any man. But that is not how it works, because people have a right to agree to being touched/sex with people that they fell comfortable/attracted to and refuse with people they don't want to do those things.

The real double standard is to deny women the right to decide that they don't want a man to touch them because that man feels he should be treated the same as the man she finds comfortable and attracted to regadless of her preferences.

The abuse suffered by Brendar Fraser is an example of this happening to a man. No one considers that he is applying a double standard because he might have allowed a lover grab his ass at some point.

Don't touch other without making sure they are comfortable with you touching them. Many people manage to do that without problem, there is no reason to creare a double standard for the people who refuse to respect that.