I agree...if that feeling isn't there anymore and you both have confessed, it's not your responsibility to look after his needs over yours anymore. You could help by maybe taking him to see someone...

I had a first like that as well, except he would watch movies and play video games with his brothers or be studying on the computer. He also worked part-time as well. But same issue it felt like it...

I think the above comment is normal. But not so if the animal is replacing the child and the child is therefore neglected because the animal is easier to care for.