Is it normal to wear a sweater without something below?

I often wear sweaters without a shirt under it. Especially in the time between spring and summer when its too hot for both t shirt and sweater and its too cold for only a t-shirt. Recently a friend of mine said thats ridiculous and only normal if you wear like only a sweater when going to sleep. That made me a little insecure and now I ask myself if its normal to wear a sweater in public without something under it.