Is it normal to delete someone's phone number after they've rejected you?

Yes, it is normal. It is healthy for both you and her to just get rid of her number.