Non exclusive relationship.. turned into feelings.

I'm 18 and I'm sleeping with a guy in college. He's 24. We agreed to be very non exclusive, not telling anyone, and not really talking to each other in classes. The sex is great, but afterwards, he tends to stay around to have conversations. That is fine, but now I'm starting to develop feelings for him. I told him about this, and he said to not have feelings for him, and he still wants to come over and have sex... part of me wants it, and part of me doesn't. I don't know what to do...