Is it normal to have sexual tension amongst male friends?

So I recently have discovered that I'm bisexual and want to explore it more. I have been hanging out with this new guy who, for all I know, is completely straight. We talk about girls a lot and he has pictures of hot girls on his phone and whatnot.

But whenever we hang out, we always sit really close to each other. I have a big couch that fits 3 comfrotably, but no matter what, we end up with our knees/shoulders touching and kind of leaning into each other. My female roommate commented the other day that she thinks there a sexual tension between us.

Then, this past Friday night, we got drunk and passed out on each other on the couch. In my drunkenness, I remember feeling him up in the morning and stroking him from outside his pants while I think he was asleep. The more I think about it though, I feel like he had to have been awake and let me do it. I have no idea though, I was still significantly drunk.

I'm really interested in seeing if there's anything there but not sure how to proceed. What does everyone think? Normal to have this tension with a friend? Am I reading too much into it? Or is there something there? Any advice on how to proceed would be helpful.