Were the actions of this woman morally justified?

I don't think so. She ought to have kept contacting the police until they eventually listened to her.

For all she knew, the harasser could have been a prank caller.

You say this is about the woman's actions, not the contractor's. I say because the woman hired the contractor knowing the contractor's ability and intentions, she is morally responsible for everything he did; the two are bound together. I believe killing a man without a trial is always unjust, and she is as responsible for his murder as the contractor is.