Is it normal to want to break the ice? will we ever?

So, there's this man I noticed around work (same building but *not* in the same workplace). I was seeing him around a few times a week, on breaks with his buddies, and he would always try to catch my eye when we passed by each other. My intuition about these things is usually pretty good and I was 90% certain we were mutually interested in getting to know each other. Work friends helped me find out his name, whether he's single (he is but probably divorced) and suggested that I should email him and ask him for coffee. He replied to me and said here is my contact info for you. But after I called his work cell and left a message, he never called back. Too complicated to explain here but I know he knows who I am and that it was me who asked him for coffee. And he still gives me the strongest impression that he's looking at me and that he is actually interested. But for some reason he doesn't approach me. Most of the time, he's with his buddies, but I could really use some help decoding this guy. If he really wasn't interested, why didn't he reply to my initial email saying, "I'm flattered, but I just started seeing someone" or something like that? At this point, I feel that to approach him or contact him again would just look desperate. But I continue to find him attractive and neither of us is a spring chicken. Is there any way to encourage him to approach me or is it too late? Lots of comments, please!