Imagine checking an air conditioning system with a pressure gauge that’s consistently 15 PSI too high. You might make the wrong diagnosis, replace parts that don’t need replacing, or overlook the actual problem.
The gauge gives you a reading, but that doesn’t make the reading accurate.
Customer satisfaction surveys work much the same way.
Every question, rating scale, and response option influences the information you collect. A poorly designed survey can give you encouraging numbers while concealing serious problems. A well-designed survey can reveal weaknesses, identify outstanding technicians, improve customer relationships, and help you make better business decisions.
And there’s an important difference between creating a survey that produces positive answers and creating one that accurately measures positive experiences.
The second approach is far more valuable.
Fortunately, survey design isn’t guesswork. Researchers have spent decades studying how people interpret questions, make decisions, and respond to surveys. Organizations such as the Pew Research Center and the American Association for Public Opinion Research (AAPOR) have published extensive guidance on the subject.
Let’s examine that science and apply it to something familiar: the residential service call.
📥Turn Customer Feedback into Better Service! Stop guessing what your customers think. Download our FREE, ready-to-use, Post-Service Survey, Customer Message Templates, and Scorecard to help you measure satisfaction, spot unresolved problems, and identify opportunities to improve your team’s performance.
The first principle of scientific measurement is knowing what you want to measure.
Consider an HVAC technician who diagnoses a failed capacitor, explains the repair, obtains approval, replaces the part, and restores cooling.
What do you want to know about that visit?
Was the customer satisfied? Did the technician communicate clearly? Was the problem resolved? Did the customer feel the work was worth the price?
These are related questions, but they aren’t the same question.
A customer could be delighted with the technician but dissatisfied with the price. Another customer might appreciate the price but feel the technician was rushed or discourteous.
If you ask only whether the customer was happy, you won’t know what contributed to the answer.
Researchers call this construct validity. In simple terms, it means measuring the thing you actually intend to measure.
For service contractors, a good survey should distinguish among overall satisfaction, technician performance, communication, and the outcome of the service call.
You don’t need twenty questions to do this. You need a few carefully chosen ones.
One of the most important discoveries in survey research is that small changes in wording can produce meaningful differences in responses.
The Pew Research Center has documented numerous examples of this phenomenon.
Consider these two questions:
Question A: “How satisfied were you with our technician’s outstanding service today?”
Question B: “How would you rate the service you received today?”
The first question contains an assumption: the service was outstanding. It subtly encourages the customer to agree with that assessment.
By contrast, the second question allows the customer to make an independent judgment.
Here’s another example.
Leading: “Did our friendly and knowledgeable technician explain everything clearly?”
Neutral: “How clearly did your technician explain the work performed?”
The first question asks the customer to evaluate several things at once, while also suggesting that the technician was friendly and knowledgeable.
The second measures one specific aspect of the visit.
This is especially important when questions are used to evaluate employee performance or determine compensation.
⚛️The scientific principle: Use neutral language, avoid assumptions, and measure one thing at a time.
A question that consistently produces a positive answer isn’t necessarily a good question. It may simply be a biased instrument.
There’s a reason technicians prefer a few meaningful diagnostic readings over collecting every possible measurement on every service call.
Efficiency matters, and the same applies to customer surveys.
In a 2009 study published in Public Opinion Quarterly, researchers Mirta Galesic and Michael Bosnjak examined the effects of questionnaire length. Participants were presented with expected survey durations of 10, 20, or 30 minutes.
The researchers found that longer advertised durations reduced the likelihood that people would start and complete the survey.
They also observed deterioration in some measures of response quality as people progressed through the questionnaire. Later questions received faster, shorter, and more uniform answers.
Think about the implications.
Adding more questions doesn’t necessarily mean you’ll receive more useful information. You might receive fewer completed surveys and lower-quality responses.
For a residential service call, I suggest starting with approximately four or five questions, including one optional written comment.
Aim for approximately one minute of completion time, and verify that claim by testing the survey with actual users. One minute is a practical design goal, not a scientifically established universal limit.
There’s no reason to ask a homeowner fifteen questions about a capacitor replacement when five thoughtful questions can tell you what you need to know.
⚛️The scientific principle: Minimize the effort required to respond while preserving the information needed to make decisions.
Question order is another well-established source of survey bias. Researchers call these order effects.
Suppose you begin your survey with the question:
“Were you satisfied with the price of your repair?”
Then you ask:
“Overall, how satisfied were you with our service?”
You’ve just encouraged the customer to think about price immediately before evaluating the entire experience.
For a customer who thought the repair was expensive, that sequence might influence their overall rating.
Now consider asking first:
“Overall, how would you rate the service you received?”
This allows the customer to make a general assessment before being directed toward specific aspects of the experience.
Pew Research Center and AAPOR both discuss the importance of question order and commonly recommend asking broad questions before related detailed questions.
For most service companies, I suggest making the overall service rating the first question.
Follow it with questions about the technician, communication, and outcome of the repair.
⚛️The scientific principle: Measure the overall impression before introducing specific details that might influence it.
A five-point rating scale is a practical choice for residential customer surveys.
Survey-methodology research generally supports using a moderate number of response categories. Five- and seven-point scales are common because they balance meaningful distinctions with manageable complexity.
For customer service, a five-point scale also has the advantage of being familiar and easy to understand.
Consider the following:
| Rating | Meaning |
| ★ | Very Poor |
| ★★ | Poor |
| ★★★ | Fair |
| ★★★★ | Good |
| ★★★★★ | Excellent |
Notice that every rating has a clearly defined meaning.
A star system without labels can be ambiguous. One customer may consider three stars average, while another considers anything below four stars a failure.
Labels help establish a common reference.
Avoid manipulating the scale by making negative answers difficult to find, preselecting five stars, or using emotionally charged descriptions for unfavorable responses.
And once you’ve selected your scale, keep it consistent.
If you change the wording or meaning of the ratings every few months, comparing results over time becomes less reliable.
There’s no universally perfect rating scale, but consistency is essential when you want to measure improvement.
⚛️The scientific principle: Use clearly defined, balanced response options and maintain consistent measurement conditions.
Here’s a situation that deserves more attention.
A technician finishes the job and asks the homeowner to complete a survey while the technician stands there watching.
What happens?
The customer may feel uncomfortable giving a poor rating, even if something went wrong.
Researchers call this social desirability bias. People sometimes provide answers they believe will be viewed favorably by others.
Customers may also simply want to avoid an uncomfortable conversation.
One way to reduce this pressure is to let customers complete the survey privately after the service call.
An automated text message or email can provide that opportunity without requiring the technician to personally request a rating.
The invitation should explain why the feedback matters without implying that the company expects praise.
For example:
How was your service visit?
Thank you for welcoming us into your home! We’d appreciate hearing how your visit went. Your feedback helps us recognize good service and identify areas where we can improve.
The survey takes about a minute.
This introduction is friendly, respectful, and transparent.
It also welcomes both positive and negative feedback.
⚛️The scientific principle: Reduce social pressure and allow customers to report their experiences independently.
Based on these principles, here is a practical starting point for HVAC, plumbing, and electrical contractors.
Question 1: How did we do?
Response: One to five stars, with descriptive labels.
This measures the customer’s overall impression.
It’s short, easy to understand, and appropriate as the first question.
Question 2: How would you rate your technician?
Response: One to five stars.
When practical, display the technician’s name so the customer knows exactly whom they’re evaluating.
If several technicians participated, make clear which person the question concerns.
This rating provides a useful starting point for recognizing outstanding employees and identifying coaching opportunities.
However, don’t assume a low score automatically means poor technician performance. The customer may be reacting to price, scheduling, company policy, or circumstances beyond the technician’s control.
Question 3: How clearly did your technician explain the work performed?
Response: Very clearly, Somewhat clearly, Not very clearly, Not at all clearly, or Not applicable.
Communication is an important part of the customer experience.
A technician can perform an excellent repair while leaving the customer confused about what failed, what was repaired, or what to expect next.
This question can identify communication problems that might otherwise go unnoticed.
Question 4: Was the problem you called about resolved?
Response: Yes, completely; Partially; No, not yet; or Too soon to tell.
For service contractors, this may be one of the most operationally valuable questions.
A satisfied customer and a completed repair are not always the same thing.
Sometimes a technician performs the authorized work correctly, but additional repairs, parts, or follow-up visits are still necessary. Providing customers with access to a customer portal gives them a convenient way to review shared work orders, check estimate status, and stay informed about their service.
A response indicating an unresolved issue should be reviewed against the work order before concluding that the technician made a mistake.
Question 5: Is there anything else you’d like us to know about your visit?
Response: Optional written comment.
This is where you may discover things you never thought to ask.
Perhaps the technician helped an elderly customer move furniture. Perhaps the office neglected to communicate a scheduling delay. Maybe the homeowner appreciated how carefully the technician cleaned up.
Open-ended questions allow customers to raise issues outside your predefined categories.
But writing takes effort, so keep this question optional.
One additional recommendation: Make only the overall rating mandatory. Requiring answers to questions a customer cannot evaluate may produce misleading responses or cause the customer to abandon the survey.
Timing deserves careful consideration.
A survey sent months after a service call may require the customer to reconstruct details from memory.
A survey sent while the technician is still in the home can introduce social pressure.
A practical approach is to send the survey shortly after the work order is completed, when the experience is still relatively fresh and the customer has an opportunity to answer privately.
For example, a contractor might begin by sending surveys 30 to 60 minutes after technicians leave, but note that this is not a scientifically proven optimum.
The ideal timing may depend on whether the job involved an emergency repair, maintenance, an installation, or a problem requiring additional time to verify.
A customer who just had a furnace repaired can probably rate the technician’s professionalism immediately. However, they may need more time before confidently reporting whether an intermittent heating problem has been resolved.
You can test different delivery times and compare response rates and the usefulness of the answers.
That’s the scientific method applied to customer feedback.
Suppose your company completes 500 service calls in a month.
You send 500 surveys.
Only 40 customers respond, and almost everyone gives you five stars.
Are you doing an outstanding job?
Possibly. But the data alone doesn’t establish that.
You received responses from only 8% of the customers contacted.
What about the other 92%?
Perhaps they were satisfied. Others may have been indifferent. Dissatisfied customers might have ignored the survey entirely.
This is called nonresponse bias. It becomes a problem when customers who respond differ meaningfully from those who don’t.
A high average rating doesn’t automatically mean your survey accurately represents all customers.
Track at least three measurements:
Response rate: The percentage of customers who completed the survey out of those who received an invitation.
Average satisfaction rating: The average score among customers who answered the rating question.
Low-rating percentage: The percentage of respondents whose ratings fell below your chosen threshold.
For example, compare two hypothetical months:
| Measurement | Month 1 | Month 2 |
| Surveys delivered | 500 | 500 |
| Completed surveys | 40 | 150 |
| Response rate | 8% | 30% |
| Average rating | 4.9 | 4.7 |
At first glance, Month 1 looks better because its average rating is higher.
But Month 2 collected almost four times as many responses and may provide a broader picture of customer experiences.
That doesn’t prove Month 2 is less biased. It simply demonstrates why average rating and participation must be evaluated together.
Also track survey eligibility, failed deliveries, and customers who declined to participate. Consistent definitions make the results more useful.
⚛️The scientific principle: Understand who you’re measuring and who may be missing from the results.
Consistency is critical in scientific testing.
Imagine having five technicians measure refrigerant pressures using different gauges, different procedures, and different reference conditions. Comparing the results would be difficult.
The same is true of customer surveys.
If one technician asks only happy customers to complete a survey while another technician never mentions it, the resulting ratings cannot fairly be compared.
Automation helps standardize the process.
Modern field service management software can automatically send surveys after work orders reach a designated status. The same questions, rating scales, and delivery rules can be applied consistently.
The software can also associate feedback with the correct customer, technician, work order, and service date.
This is useful for several reasons.
First, survey delivery no longer depends on whether the technician remembers to request feedback.
Second, management can evaluate results across technicians, departments, and time periods.
Third, low ratings can trigger an internal notification or follow-up task.
Fourth, positive comments can help managers identify employees whose work deserves recognition.
Automation doesn’t guarantee unbiased results. You still need to use reasonable selection rules, accommodate customer communication preferences, and examine response patterns.
But it can eliminate much of the inconsistency caused by manual survey administration.
In the service business, small problems can turn into expensive ones when they go unnoticed.
A customer who says the technician didn’t explain the repair may simply need a follow-up phone call.
A customer who reports that the original problem remains unresolved may need another service appointment.
Someone who describes an unusually poor experience may need personal attention from management.
This is where a survey becomes more than a scorecard.
Consider setting a rule that alerts management whenever an overall rating falls to three stars or below.
The exact threshold is a business decision. The important part is that the rule is consistently applied and someone is responsible for following up.
You can also use specific answers to trigger different responses.
For example, a partially resolved repair might be routed to the service manager, while a low communication score could identify a coaching opportunity.
Not every poor rating requires an apology, refund, or disciplinary action. It does require understanding what happened.
A survey becomes much more valuable when its results lead to appropriate action.
Customer satisfaction surveys and online reviews serve different purposes.
An internal survey helps you understand performance, recognize employees, and identify opportunities for improvement.
A public review helps prospective customers evaluate your business.
Both can be valuable, but they shouldn’t be confused.
Some companies attempt to improve their online ratings by inviting only customers who gave five stars on an internal survey to post a public review.
This practice is commonly called review gating.
Google’s review policies prohibit businesses from selectively soliciting positive reviews or discouraging negative ones.
A better practice is to make review opportunities available without filtering customers according to their survey ratings.
Ask for honest feedback, not a predetermined score.
You can still contact dissatisfied customers to resolve problems. In fact, responding appropriately to criticism is one of the most important uses of customer feedback.
But don’t make eligibility for a public review depend on whether the customer has given you a favorable internal rating.
Once your survey is operating, don’t assume the initial design is perfect.
Test it.
You might want to know whether customers respond more frequently to a text message or an email.
Or perhaps you’re wondering whether customers respond better 30 minutes after a call or three hours later.
Rather than guessing, conduct a simple controlled experiment.
Randomly divide eligible customers into two groups.
Send one group version A and the other group version B. Keep the questions and other conditions the same, changing only the factor you want to test.
Then compare the results.
For example:
Compare response rates, survey completion rates, and the quality of the information collected.
Make sure you have enough responses to distinguish a meaningful difference from random variation. Small differences in small samples are often inconclusive.
Also recognize that if you change the wording of a rating question, the resulting scores may no longer be directly comparable with previous surveys.
This kind of testing is called an A/B test, and it’s widely used in marketing and software development.
It’s also an excellent way to improve customer surveys without relying on assumptions.
A successful survey isn’t necessarily the one that produces the highest average rating.
It’s the one that helps you understand your customers and make better decisions.
Consider what you can learn from a well-designed program:
The greatest benefit isn’t the survey score itself.
It’s what you do with the information.
Technicians understand the importance of accurate measurements.
You wouldn’t intentionally calibrate your instruments to show what you want to see. You calibrate them to show what’s actually happening.
Customer satisfaction surveys deserve the same approach.
Use clear, neutral questions. Keep the survey short. Ask about one thing at a time. Use consistent scales. Make it easy for customers to respond privately and honestly.
Send the survey consistently, monitor participation, and take appropriate action when the results reveal a problem.
And remember that scientifically informed survey design is not about tricking customers into giving you better ratings.
It’s about removing obstacles and biases so that good service receives the recognition it deserves, while problems become easier to identify and correct.
That’s how a simple “How did we do?” survey becomes a valuable business management instrument.
For a routine residential service call, four or five questions are a practical starting point. The best number depends on the information needed and the effort required to answer. Keep the survey short enough that customers can complete it conveniently.
A five-point scale is a sound choice for a short post-service survey. Clearly label the response options and maintain the same scale over time so results can be compared consistently.
A useful starting point is shortly after the technician completes the service call and leaves the home. Test different delivery times to determine which produces the most useful responses. Jobs requiring follow-up testing may justify a later question about whether the problem was resolved.
Yes. Excluding dissatisfied customers creates selection bias and can hide important performance problems. An honest survey process should include all eligible customers under consistent rules.
You can invite customers to share honest reviews, but you should not selectively invite only customers who gave favorable internal ratings. Follow the policies of the review platform.
The recommendations in this article draw on established survey-methodology research and professional guidelines.
Pew Research Center — Writing Survey Questions. Explains the effects of question wording, question order, and questionnaire design on survey accuracy.
https://www.pewresearch.org/writing-survey-questions/
American Association for Public Opinion Research — Best Practices for Survey Research. Covers unbiased question wording, sampling, response options, survey testing, data quality, and responsible interpretation.
https://aapor.org/standards-and-ethics/best-practices/
Galesic, M., and Bosnjak, M. (2009). “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey.” Public Opinion Quarterly, 73(2), 349–360.
https://doi.org/10.1093/poq/nfp031
Educator’s Blueprint: A How-To Guide for Survey Design (2022). Peer-reviewed overview of survey construction, response scales, and question design.
https://pmc.ncbi.nlm.nih.gov/articles/PMC9399445/
Google — Maps User Contributed Content Policy. Official policy covering review authenticity, rating manipulation, and selective solicitation.
https://support.google.com/contributionpolicy/answer/7400114
Automated text messages are a convenient way to communicate with customers, send appointment reminders, and keep everyone informed about scheduled service work. For HVAC, plumbing,
What Is AIA Billing and AIA Payment Applications? AIA billing is a standardized method of construction billing using a percentage-of-completion method (i.e., progress billing). The
An adjusting journal entry updates account balances at the end of an accounting period. Also known as an adjusting entry or AJE, it ensures that
A Customer Survey Is a Diagnostic Tool. Are You Getting Accurate Readings? Imagine checking an air conditioning system with a pressure gauge that’s consistently 15
By submitting this form, I agree to receive marketing communication via phone call or email from Aptora Corporation.
By submitting this form, I agree to receive marketing communication via phone call, or email, from Aptora.
