Is it normal to date my best friend's ex?

Ok so he's not actually her ex it was more like casually dating, first they went on one date and a couple of weeks after (he didn't text her at all in that time) he invited her to his friend's house and i went with her, my first thought when I meet him was that he was cute and nice but that was all, I spent the night talking with him and his friend while my best friend ended up talking about her ex (it was a recent break up), that night ended with my best friend and the guy going upstairs to have sex and me kissing his friend.
Like a week after that they went on another date and my friend invited me to the pool with him, on the way over there she told me how he told her that it was better if they didn't see each other anymore because he got the feeling she was still hung up on her ex (totally true) and he just didn't feel that much chemistry with her AND he also told her that he liked me and wanted to know if she was okey if he asked me out.
My first reaction was like of course not, 'cause I knew that she liked him and a had a great time with him even though she told me it didn't matter if I liked him. So we went to the pool but now with that knowledge I started to see the guy differently and with my friends permission we started going out, at first it was just pretty casual but I did see the differences between how he treats me and how he treated her, we talk on the phone everyday and see each other every weekend I even stay at his house a few nights a week.
The thing is I'm starting to fall for him, and while it was ok when things were casual I'm not sure about the idea of seriously dating someone who has been with my best friend and I've noticed that she's uncomfortable with the idea of him and I spending so much time together even if she denies it.
This is also the first guy i slept with who i wasn't seriously dating or at least seeing for a couple of months, can I even trust a guy who jumps from one friend to another? (He says this is the first time he does something like that but one can't never really know)
Is this whole situation normal? What should I do?