He made a joke that went too far: how to respond

The first thing to sort out is what the joke was pointed at. A joke about a shared thing, like a bad restaurant or your terrible sense of direction, is a different animal from one that pokes at something you didn't choose or that you're already insecure about. Ask yourself whether it was about you as a person or about a habit, and whether the audience mattered. Jokes made in front of other people carry extra weight because they also set the status between you.

Also notice what he did after saying it. If he watched your face and adjusted, he was reading you. If he kept going and got louder, that's a different signal. The joke itself tells you less than the next two minutes do.

A good reply names what bothered you in one concrete sentence, something like the specific thing he said and how it felt. Keep it about the moment rather than about what kind of person he is. You want him to hear the feedback, not defend his whole character. Say it the way you'd say it to a friend, without hedging so much that the point disappears.

Then stop. Don't add three follow-up complaints or a summary of everything he's done wrong. One clear sentence gives him something specific to respond to, and a vague accusation gives him nothing to do except argue.

Once you've said it, his answer tells you the most. Owning it means he acknowledges the specific thing, doesn't pile on excuses, and maybe asks how it came across. Managing it means he apologizes for your feelings instead of his action, says you're too sensitive, turns it into a joke again, or goes quiet in a way that makes you feel like the problem.

You don't need him to grovel. A plain acknowledgment is enough. If you're getting deflection, that's useful information even if the joke itself was small.