Leave Me Alone, Hasidics

While walking down the streets of Park Slope on my way to teach a class, I was waylaid by a Hasidic man and his son who shouted after me, inquiring if I was Jewish. I wanted to yell back, "Yea, I am," and, without breaking stride, "are you?" This would also be accompanied by a cold stare into his puzzled eyes. Instead, I settled for cursing at him when safely out of ear shot. I don't understand how standing on a street corner and asking passerby who look like they might be Jewish (nose and shirt that says "I may be Jewish, ask me" give it away every time) will help you get in ANYONE'S good graces. Maybe the mitzvah shouldn't be for saving people, but for not annoying them instead.