My $.02 as someone who spent a few years making up questions for people to ask at hotels….
1. You need to have questions to ask the staff so you can gauge their response.
2. The question has to be easy enough so that any staff member could take the initiative to answer, rather than directing you to the concierge.
3. Having the same question asked at all properties in a particular month/quarter levels the playing field for overall scoring.
4. Sometimes the stupidest questions are the best test for the staff.
After a few months of making up hotel instructions for a living, I started running out of ideas for questions to ask. Some days, I would take my laptop, get a cup of coffee and work from a nearby hotel lobby where I would eavesdrop on each question asked of the hotel staff. I learned a lot from that. If you get hotel instructions from the MSC I used to work for, they come from a database that was complied mainly from questions that I actually heard guests ask.
The main thing I learned from watching guests in hotels and talking to the hotel management was that it's not as important what the answer to the question is, but instead how the request is handled. A doorman will get asked where the nearest Starbucks is 10 times each day…even if the local Starbucks is in clear view from the hotel lobby. Does he roll his eyes at the question or answer respectfully?
As much as a MSC would like to have their questions customized for each hotel. It's just not possible. I would try to catch obvious stupidity (Like a wine request for a hotel in Napa), but some days I was sending out dozens of instruction sets for hotels all over the world. I prided myself on knowing the properties, but they changed so fast that it was not possible to keep up with all of them.
That said, it's your responsibility as an evaluator to make a intelligent decision about the questions presented to you and contact the MSC if you do not feel they are appropriate for the property you are evaluating.