Keep a boyfriend or lose a friend

I have been friends with Bob since I was in high school. We have not seen each other for two years because he and I were both going through personal problems; him with his girlfriend and me with stupid guys. I consider him a family because he has helped me in my most difficult time. He is still friends with my sister and she told me that he wanted to talk to me. He told me that he is sorry for getting mad at me and for avoiding me for so long. He asked me to forgive him and that he wanted to continue being friends. I told my boyfriend of three years this and he said he does not want me to be friends with him because:
1. He hung out two times with a guy I used to see
2. He thinks he has a crush on me
3. He thinks Bob is going to invite his friends and they are going to hit on me and I am going to cheat on him.
4. I am disrespecting him if I start talking to him.

I got mad hearing this because he has best friends who are girls and I am ok with him hanging out with them. He even kissed one of them and he still talks to her and I do not care because it has been years since they kissed and I know it meant nothing. I told him if I cannot be friends with Bob that he can't be friends with his friend girls and he said ok but I know he is lying because they are family to him and they have been friends since high school.

I do not know what to do? I know I am not doing anything wrong because:
1. I am not going to hang out with Bob more than my boyfriend
2.I am not going to cheat on him because c'mon I see Bob as a brother and if he even hits on me I would yell at him that we cannot see each other anymore because I have a boyfriend and I respect him.
3. I am not hanging out with Bob alone, only if my sister comes along.

I am confused because even though I know I am doing no wrong my boyfriend says I am and that I am disrespecting him. We have been arguing about this issue for a long time and he made me choose him or Bob. I got frustrated with him and I ended the relationship because I feel he has no trust in me. He yelled at me and said that I choose Bob over him when I am just tired of him assuming that I am going to cheat on him. He continues to tell me that I ruined the relationship because of Bob. I want to see other people views on this.