If you're a woman, you can't.
If you're a dude, yes.
If you're a dude who happens to be on the heavier side, you can, but you risk others making fun of you. I say this because I've seen dudes on the heavier side get ridiculed for being shirtless.
Sure you can until you need to go to the bank, supermarket, work or any number of other places.
But what I don't get is why a skinny little white skinned runt like you wants to go shirtless. I'd think you'd be tired of everybody laughing at you behind your back, by now.