Is it normal to feel this guilty after rejecting someone?

I think having these feelings in this situation is okay. Sometimes experiencing those unpleasant emotions lets you know that you're human.

The best thing you can do is realize you have to do what is right for you; no one else is going to watch out for you or make the best decisions for yourself but you. So, overcome these feelings and have confidence in your decision. It's the only way.