People think you are guilty because you are upset you were accused.
I hate when someone says something about you that isn't true, and then if you get upset they will say something like "well if it isn't true, why are you getting so upset?" as if that somehow proves it really is true. Um, I am getting upset because you are saying terrible lies about me right to my face! Also, if you don't get upset it goes the other way and they say, "You don't seem too upset or surprised about being accused." So if you get angry about it, that proves you did it, and if you don't, that also proves you did it, you can't win.
All that is really happening is the accuser believes it no matter what you say, and is just justifying their beliefs to themselves. But in reality, they have proved nothing.