Does everyone hear "the sound of silence"?

I do hear it anywhere quiet enough, but it's so quiet that it really doesn't affect my hearing, and is just annoying.

I'm assuming that what I hear is just some system in my body like blood flowing through my brains