Is it normal that i need to always have a war with volume?

I'm 19 years old, and I constantly play music. Not very loud, I know for a fact. I know it's low enough to where people won't hear it, but there's just... Something weird about headphones with me. The isolation in my ears makes me feel like at any volume (even it's lowest) that others will hear it. Even when I reassure myself that it's low enough where nobody will hear it, I still have a need to always continue to lower it.... Then raise it when I'm reassured.... Then lower.... Rise.... Etc.

IIN?