Is it normal I get irritated when I hear people imitate accentst?

Unfortunately derker is right. People are just ignorant and if this annoys you the next time, the only thing you can do is correct them however you see fit. Angrily or politely.