Is it normal to, without a doubt, hate it when people call you rude?

I like being a gentleman, I barely have any problem being polite now, but refusing to say please is no longer a problem, you don't have to say please, a simple 'yes ma'am', or 'thank you' is enough. But there's a thin line between polite and failing at it. But when they say about me "he's so rude" I absolutely hate that, yet I'm not rude about it, I simply say it's rude to call people rude and a true gentleman doesn't label people that, this is why people hate those people. But here I am, would such a thing be normal in polite society?