Is it normal to be scared of sex

im 20 and im perfectly fine down there but im so intimidated by the whole thing. im shy and akward socially i dont have the confidence and the funny thing is im a 6'4" 200 pound kid who works out and is in great shape. i had my fair share of hook ups threw out high school ya know make outs handful of BJ's and most of it came when i was drunk. the couple time i did have a chance to have sex we either didnt have a condom or i got wiskey dick and one time i couldnt get hard when i was sober cause i think i was so nervous but i dont even feel that nervous. of course i was a little but not enough that i think i couldnt get it up. anyways as time has gone on i have build up this brick wall that i cant get passed..
any suggestions??