I think you handled the situation well. Our job is to report facts. It's not our job to aim to get someone into trouble, nor it is our job to aim to get someone a bonus for a good job. The facts should speak for themselves.
I believe there should always be room for grace. People do have "off" days. You never know that he might have gotten a bad doctor's report just prior to picking you up. I am in a CRAPPY mood when I am stressed due to something bad going on. Some of you have witnessed that. I try to extend that if someone is rude. If it is only one time, I will ignore it or am more apt to think it was just my perception.
I'm mostly in agreement with you, the "rudeness" would have bothered me less than taking me to the wrong destination. Getting someone to their destination, ideally on-time, IS the job. Given his age and the fact you picked up on he could have something medically going on, I think it would be relevant to mention it in the report. I'm not familiar with this shop to know whether there was a question where the issue could have tied in. I'm not clear on exactly what you said regarding that.
As far as feeling bad, I wouldn't. At least not for writing an honest report. It's okay to feel bad for him because of his age and possible medical issues. If your report gets him into trouble, even fired, it not on you. Letting him go because he could be a liability could save an innocent person's life, if he shouldn't be driving.
Whatever the result of your report, you are not responsible for it. You did your job and, from where I sit, you did it quite well. The client is responsible for the final outcome. IMHO, I don't think your report would sway one way or the other, based on what you described. I think it could be used to confirm actual customer reviews or complaints. If they see a trend, that is how a report like this could play a role.
There is really only one shop where I've felt bad for doing the right thing. There is a very nice middle-aged man who owns a EM location about an hour from me. He does not have a c-store, but he does have 2-stall garage, along with a couple of pumps for gas. His station is the worst looking EM out of any gas station (at least 100 individual locations) I have audited. Everything element is severely faded, cracked, the pumps have large dents in them, broken light fixtures, and the list goes on. Assuming he has the same deal as the other EM's I audited, I know he gets fined for not being in compliance. I'm sure the upgrades would be well over 100k. I feel bad for him because he has no employees. He makes his money working on cars. I had a customer tell me that he does the best car work than anybody around, and that he keeps his prices affordable for the low-income people in the area. I've not audited him since December, but I still can tell you his name, that's how much he sticks out in my mind. Month after month though, I would have to snap pictures of all the elements not in compliance, and mark it on the report. I am guessing he does not drop the EM branding because it would still require him to make upgrades. Some of the elements can be painted over, but I'm not sure about the actual gas pumps, whether they would have to be replaced.