Is it normal that i'm afraid to orgasm?

I'm a twenty-year-old guy. Seriously late bloomer. Like, I was 19 before getting facial hair. Virgin, but not because of lack of opportunity. On the contrary, I've chickened out twice. I tried things on my own (a while ago...years maybe), and every time I got close to climaxing, I got overwhelmed by involuntary anxiety, controled my breathing, and avoided the orgasm. Maybe it's because I tried as a young kid and it hurt. I don't know, it's totally involuntary. But I'd like to eventually have a child, so this is a problematic issue. I already have no real sex drive, so I haven't really had an incentive to try to straighten this out...