Is it normal if sensitive people annoy me?

Okay I just can't stand being around really sensitive and serious people. I feel this pressure where I have to behave every min or else they're gonna judge me or I'm going to get looked down on, even for things that are not that bad. To others it's whatever and I love those people for not thinking too much of it but to sensitive freaks, they take it to a whole other level. I don't feel free, I feel like the world is about to break because they just got their feelings hurt or they quickly jumped to conclusions about something. I really wouldn't give this much of a damn but unfortunately, I'm surrounded by a lot of those people especially family. What annoys me even more is the double standard of it. According to them, they can do a certain thing and justify it as innocent because it's them and they "mean well" while others can't because they're coming from a bad place, or really that's how they perceive it ass when really they are just as innocent. When I was younger, I didn't care as much but it did lead me to being known as this "bad person" or just the more "outspoken one." Now Im older and I care more about what others think cause thats just what happens when your grow older but in a way I wanna not care anymore cause it's so stressful holding back just so you don't hear any bitching from the other side. As long as you're behaving like they want you to, then everything is fine. It shouldn't be that way. How do I deal with these types of people? I have been clear on how sensitive and serious they are but they surprisingly get hurt and respond with this stubborn attitude like their belief is right and that's it. I like hanging out with free spirit people who mean well but don't things too seriously and basically stand on the same page as you do.