When do you stop fighting for your love?

hey there, I'm looking for some advice on my current confusing relationship. Before meeting each other, we were both single for a while. We had our own goals and things we wanted to do but when we met we just dove right into it. We live about an hour apart and started off seeing each other at least once a week, slowly it became more frequent. About 5 months in he brought up the idea of moving in with each other. I thought it was a great idea and did some research and set up appointments. The day of the appointments, he told me he we was not ready to move away from his friends and family yet. I understood but was a little upset. We were so busy doing fun things all the time and even went on a road trip across country, which was amazing! About 9 months into dating, we still had not told each other "I love you". Although it was clear we both did love and care about each other (I was being too stubborn to say it because of a previous relationship). I kind of brought it up one day and he got weird and went home uneasy. Later on in the day he texted me this long text saying how. much of an amazing person I was, and how much he cared about, and that he was in love with me. He said he never felt this way and wanted to take a break. I was floored! I called him and we had a long conversation about our feelings and I told him I felt the same and he shouldn't push me away, that he should be happy with these feelings. We continued to stay together and had the most amazing relationship. Everything was perfect. We did a lot of things together, things with our friends, things with our families, and were also able to give to give each other freedom to do things on our own. He decided to go on a spontaneous trip to visit his friend who lives on his own in a different state. He asked me to go but I was unable to due to work. He went anyway and we were talking/texting like usual and I was thinking how great it was that we can have a relationship where we totally trust each other to be on our own. He came over after he had come back from his trip and broke up with me. I was completely shocked, he was upset and crying as well but said this is what he needs. A miserable week went by and we decided to meet and talk about it more. He said he wasn't looking for anyone else, and that he loved me but there was goals he had before he met me(like live on his own) that he felt he couldn't accomplish if he was in a relationship with me. I told him I couldn't be his friend because I was too in love with him. We agreed to have no contact until he "wanted to contact me". After 24 days he texted me and made small talk asking me "how things are?". I kept it short and asked if he wanted to get together. He said he did and I texted him back saying I was still not interested in just being friends with him, and maybe we could go out to dinner on a date, nothing serious, or committal. He said "I would like that.". About  week later we met up and hit it off right away with great conversation and jokes! We went out to dinner, and he asked to split the bill. That's when I knew that he just saw this as hanging out. I thought I had been clear with him on my feelings so I felt a little embarrassed when asked to split the check. Afterwards, we were sitting in his car and I asked what he'd been thinking about us. He started to get upset again and said he hadn't changed his mind and he still didn't want to be in a committed boyfriend/girlfriend relationship. I told him how I thought im how I thought he was my soul mate and he said he felt the same but he "can't change his mind". I told him I wouldn't be his friend and we agreed there wasn't really a label for what we are. He took my hand and told me that we'd figure it out. We hugged for a long time, which turned to kissing, which turned to having sex, twice. Great sex! We agreed to keep it really honest. We've been talking everyday but its only ever in the time frame that he's at work. I feel really confused. I'm making an effort to be happy on my own, . I'm making an effort to be happy on my own, and I am, but I can't get him off my mind. I hate this thing with him only texting me during a certain time frame. I know that I'm not going to able to just be friends with him. Should I take it slow and see where this in between friends/friends with benefits/relationship leads to? Or should I put my foot down and tell him I can't be his friend and If we can't be dating I can't talk to him? He's so important and special in my life and I would hate to lose someone as special as him . I hope he's not just hoping my feelings will fade to friendship, because they will never be that way and I don't know how to make that clear to me. Does he just need time or does he want the bachelor life?