Is it normal to get upset at people who abuse handicap permits?

Agreed--unless the person is driving someone else who IS clearly handicapped. But, it sounds like that isn't the situation your'e describing.