Is it normal that he seems to like me but won't bring it up?

I'm a 32-year-old single girl. About a month ago, I started seeing a guy I've been interested in (roughly the same age). There's a 3-hour distance between us, so we've been on only 3 dates, if you could even call them that. They've been very casual, which is okay at this point. He's not an active pursuer who tells me how he feels and lines up the next date at the end of the current one. But he's very sweet and funny, and opens the car door for me. When we're not on a "date", he's texting me multiple times nearly every day!

My problem is, how do I proceed from here? I'd like this to develop into a relationship if possible. He admitted that he's "not a planner" and never has been one. As a result, I'm left to wonder, each week, if I'll see him that weekend. The last time we went out he suggested it, but he knew I would already be in his area having a girls' night the night before. I feel a bit like a fling that he wants only when it's convenient for him. I've been advised not to ask how he feels or what he wants, but it kills me to be in constant limbo. I'm used to the aggressive pursuers who make their feelings clear. I've also considered the possibility that he's an introvert (except that introverts plan and he doesn't). In addition to the distance, our schedules are kinda opposite (he has weekends off and I generally don't).

Is he just using me for sex (which is mind-blowingly great)? Do I just need to give it more time? Should I ask him about it? Or should I forget about him and move on?

There are virtually no other romantic prospects where I live, and the guy in question is a fantastic catch I'd hate to "lose". Help! I really like him.