How do you confront someone who uses that passive aggressive

How do you confront someone who uses that passive aggressive tactic of "just kidding" "just joking" when they are really NOT.

I have found honestly to be the best tactic, "when you say that, it is still hurtful, is that your intention?"

I tell them it's not a joke to me .
My son has a habit of doing that to me. It's like he knows its will upset me. I get off phone and tell him when he wants to be nice he can call me back later

I agree! Being honest because if not they are just going to keep doing it and it's going to bother you more and more.