Boyfriend masturbates to my voice

I met my current boyfriend through a mutual friend and me and my boyfriend have a lot of similar interests so relationship-wise I feel like we're doing amazingly. He's very patient with me and I feel very safe and comfortable talking to him, something I haven't felt with previous relationships, but now I feel a bit twisted due to sexual frustrations.
I'm a virgin and it's really difficult to excite me if I'm not interested in budging and he's been made aware of this however it's clear that he has obvious sexual needs. I'm not prepared to help him due to both distance and not knowing what to do. He's recently started asking me just to talk as he masturbates to my voice and while it doesn't bother me I feel like it should? Am I weird or am I just that comfortable with him?
I don't get turned on by him doing so, but it makes me feel at ease knowing that just me talking to him is enough to help please him.