Is it normal That people who won't look anyone in the face annoys me?

Yes, it's normal. People raised in western societies need eye contact and feel threatened when they don't receive it.