Is my ex boyfriend over me? why the anger?

I have been with a man for a little over a year - on and off for the last 6 months - I am 32 wks pregnant and he denies this is his child so has asked for paternity. I said fine because the fact of the matter is, I was attacked the same month that I conceived.. We broke up in the summer - he broke up with me, I went out for a drink which I never normally do to get over him, in the process of which I got talking to a stranger and I think he spiked my drink as I was unconcious soon after - when I woke up I saw him having sex with me, started crying and got v upset and tried to report him to the police. He fled the country the next day and I felt traumatised - still do. I took an emergency contraceptive pill right after in case. Period was July 4th, rape was 8th, sex with the ex was 20th (day 16 - was ovulating too) and preg test said positive on 1st Aug. but it said 2-3 wks ago I conceived on the fancy tester. I came clean about the incident to my ex cos I didn't want to deceive him for the pregnancy over what happened but it took me 2 wks to pluck up the courage to say all.. towards the end of pregnancy he is now trying to say that I lied to him 'for 2 wks' but I haven't. I said to him I didnt have to tell you I was raped but I still did cos I thought he had a right to know and I wanted him to understand why I was having probs at the time to suddenly get close to him again when we got back with each other. For practically all of this pregnancy - all he has cared about is for a paternity test. I never made an issue about it unless he upset me or had annoyed me... so do not know why he keeps to go on about it even towards the end of pregnancy. He told me if its not his child he doesn't want to stick around and I said fine that it was upto him..and not my decision. I didn't mind if that is what he wanted to do... though it cut me up cos I actually do love him properly. He has been more and more cruel over time and we broke up as he appeared to be dating others while with me. I bumped into him a couple of days ago by accident as we work in the same place. The conversation sadly degenerated. He said he had very little trust in me and I said back that then HE had the problem. The baby is made now... and its just filling out with weight but instead he has made me miserable. He is now not talking to me and avoiding me. Im not talking to him cos think the baby freaks him out and plus he doesn't want to.