Is it normal that i can tell when people talk about me?

I hate gossipers, a lot. They're an everyday annoyance for me. The thing is, I can always know what anybody's talking about me, usually because they make it obvious and I can either hear it or see it or both. But because I find out stuff really quickly this way, people just freak out and think I'm a stalker. Why? What characteristics align me with that of a stalker? Do I just have good observation skills? What do you think?