Is it normal that i hate when people think i have mental issues?

I hate them solely because they are so quick to judge me, especially everywhere I am seen as this " deranged " person or someone who is " mentally ill ", not only do I not like named it but its just people like that are the reasons why I walk away because I know they are so quick to judge. They often think I'm mentally unstable just because I taking things very literally and taking this in differently in which I always do. Like I know beating them up wouldn't be the right answer, as much as I still do keep going to therapy at school, my therapist always told me I am normal but we're not born perfect. This was partly the reason I avoid people.