Do amputations really matter that much?

I'm amputated from the mid-calf down on my left leg from an accident rock climbing. I'm 29 and very active, do a lot of outdoorsy stuff, hiking, rock climbing, camping. So I'm in no way handicapped just cause my foot is gone. I have no self esteem issues or insecurities over it, so it's not like I come with baggage.

But, dating has been hard, a lot of people have cut me off (lol) after finding out about my leg (It's not always obvious when I'm wearing hiking boots and long pants) and it doesn't make much sense? I'm decently attractive, well off money-wise, educated, and have a lot of hobbies, it shouldn't be hard for me to find someone to get serious with. I know a lot of people find it spooky or gross. (I've had a girl ask me to cover up because she couldn't eat if she was seeing my leg) is it really that big a deal?