Sometimes they talk too fast to get a word in edgewise. When I have to ask an open-ended question, I have at least 2 in case the employee covers it. I agree, hesitation on your part is natural and shouldn't affect your report. You answered that you didn't think you've been there before. By default, that means no, right? So you provided a valid answer. I'd be more suspicious if the customer auto-answered, Yes/No with no hesitation.
___________________________________
“You can only hold a smile for so long, after that it's just teeth.” - Chuck Palahniuk