How to write unbiased survey questions (with examples)

To write unbiased survey questions, use neutral wording that does not hint at a preferred answer, ask about one thing at a time, define vague terms, offer balanced answer options with a clear opt-out, and test the questionnaire on real people before launch. Most bias comes from wording, answer options and question order, and each can be fixed.

This guide walks through ten common sources of bias. Each one has a short explanation and a before-and-after rewrite you can adapt. A review checklist closes the guide.

Why question wording matters#

A survey measures what people answer, not what they think. When the wording pushes respondents toward an answer, the result describes your question rather than your audience. That kind of response bias is hard to spot after the fact, because the data looks clean. Every row is complete and every chart renders. The distortion only shows up later, when a decision based on the data does not work out.

The good news is that most bias in questionnaires is caused by a small set of repeatable mistakes. Once you can name them, you can find them in a draft in minutes.

1. Leading questions#

A leading question signals which answer you expect. It often includes the asker's own opinion or a word of praise.

Before: "How much did you enjoy our new, faster checkout?"

After: "How would you rate the checkout process?"

The original assumes the respondent enjoyed it and that it is faster. The rewrite asks for an evaluation without suggesting one.

2. Loaded questions#

A loaded question contains an assumption that may not be true for the respondent. Anyone who disagrees with the assumption has no honest way to answer.

Before: "Where do you usually go to relax after a stressful workday?"

After: "How would you describe a typical workday for you?" followed by "What, if anything, do you do to relax after work?"

The rewrite tests the assumption first and adds "if anything" so that "nothing" is an acceptable answer.

3. Double-barreled questions#

A double-barreled question asks about two things but allows one answer. If someone rates the pair as "average", you cannot tell which part they meant.

Before: "How satisfied are you with our pricing and customer support?"

After: "How satisfied are you with our pricing?" and "How satisfied are you with our customer support?"

A quick test: if the question contains "and" or "or" between two topics, split it.

4. Absolute questions#

Absolute questions use words such as "always", "never", "every" or "all". Very few people can truthfully say yes, so answers bunch at "no" regardless of real behaviour.

Before: "Do you always read the terms before you sign up for a service?"

After: "How often do you read the terms before you sign up for a service?" with options from "Never" to "Every time".

A frequency scale captures the range that the yes/no version hides.

5. Ambiguous questions#

Ambiguous questions use words that mean different things to different people, such as "regularly", "recently" or "often". Two respondents can give the same answer while meaning very different things.

Before: "Do you exercise regularly?"

After: "In the past 7 days, on how many days did you do at least 30 minutes of physical activity?"

Replace vague quantities and time frames with specific ones that the respondent can count.

6. Jargon and acronyms#

Internal terms, technical language and acronyms confuse people who do not share your vocabulary. Confused respondents guess, skip or abandon the survey.

Before: "How satisfied are you with the SLA on your P1 tickets?"

After: "When you reported an urgent problem, how satisfied were you with how quickly we responded?"

Read each question as if you were a new customer. If a term needs explaining, use the explanation instead of the term.

7. Unbalanced answer scales#

A scale is unbalanced when it offers more positive options than negative ones, or when its labels are not symmetrical. The structure itself pushes results upward.

Before: "How was your stay?" with options: Excellent, Very good, Good, Fair.

After: "How was your stay?" with options: Very good, Good, Neither good nor poor, Poor, Very poor.

Keep the same number of options on each side, label every point, and make the labels mirror each other. For more on scale design, see the Likert scale questions guide.

8. Order effects#

The order of questions and options changes answers. A question about a recent bad experience can lower the rating of an overall satisfaction question asked right after it. In long option lists, items near the top or bottom tend to be picked more often.

Before: "How satisfied were you with the delay to your delivery?" followed by "Overall, how satisfied are you with our company?"

After: Ask "Overall, how satisfied are you with our company?" first, then ask about specific parts of the experience.

General before specific is a reliable default. For option lists that have no natural order, such as a list of features or brands, randomize the order for each respondent so position does not favour any one item.

9. Acquiescence bias#

Acquiescence bias is the tendency to agree with statements, especially when a survey is long or the respondent is tired. Agree/disagree batteries are particularly exposed to it.

Before: "The new dashboard is easy to use." (Strongly disagree to Strongly agree)

After: "How easy or difficult is the new dashboard to use?" (Very difficult to Very easy)

Item-specific scales ask respondents to think about the topic itself rather than whether to agree. If you keep an agreement battery, keep it short and watch for straight-lining.

10. Social desirability bias#

People tend to give answers that make them look good, such as over-reporting healthy habits or under-reporting rule breaking. The effect is strongest on sensitive topics and when respondents believe they can be identified.

Before: "Do you follow the company's data security policy?"

After: "Many people find it hard to follow every security rule during a busy week. In the past month, how often did you share a password with a colleague?"

Normalise the less flattering answer, ask about specific behaviour rather than identity, and tell respondents how their answers will be used. Where it is appropriate, collect responses anonymously and say so in the introduction.

Answer options need the same care#

Even with neutral question text, the answer list can introduce bias:

• Missing options. If someone's true answer is not listed, they pick the nearest one. Add "Other (please specify)" when the list may be incomplete.
• Overlapping ranges. "18-25" and "25-34" both include 25. Make ranges mutually exclusive.
• No honest opt-out. Add "Don't know" or "Not applicable" when some respondents cannot answer. Do not use it to replace a neutral midpoint.
• Forced answers on sensitive questions. Making every question required can push people to guess. Require only what you truly need.

Test before you launch#

No checklist catches everything. A pilot survey with a handful of people from your real audience shows you which questions they hesitate on, misread or skip. Ask a few pilot respondents to explain each question in their own words. If their explanation differs from your intent, rewrite the question.

Unbiased survey question checklist#

Use this list to review every question before launch:

• The question does not contain an opinion, praise or emotional words
• It makes no assumption the respondent might not share
• It asks about one thing only
• It avoids "always", "never", "all" and "every" unless they are needed
• Time frames and quantities are specific
• There is no jargon or unexplained acronym
• The answer scale is balanced and every point is labelled
• Answer options do not overlap and cover every likely answer
• There is an honest opt-out where some people cannot answer
• General questions come before specific ones
• Option order is randomized where the list has no natural sequence
• Sensitive questions make every answer feel acceptable
• The survey has been piloted with real respondents

Writing unbiased questions in Claform#

When you build a survey in Claform, a few settings support the practices above. Choice questions have a Randomize options setting in the question editor, so each respondent sees the options in a different order. On the Professional plan and above, the Randomize tool can also shuffle the order of pages and questions for each respondent. The survey test page lets you preview the questionnaire on desktop, tablet and mobile sizes before you share it with pilot respondents.

For broader advice on structure, length and flow, read the survey design best practices guide.

Frequently asked questions#

What is an example of a biased survey question?#

"How much did you enjoy our new, faster checkout?" is biased because it assumes the respondent enjoyed the checkout and that it is faster. A neutral version is "How would you rate the checkout process?"

How do you make survey questions neutral?#

Remove opinions and emotional words, avoid assumptions, ask about one topic per question, use specific time frames, and pair each question with a balanced answer scale that has a clearly labelled midpoint or opt-out.

What is the difference between a leading and a loaded question?#

A leading question suggests which answer is expected. A loaded question builds in an assumption, so respondents who do not share it cannot answer honestly. A question can be both.

Should every survey question have a neutral option?#

Not always. A neutral midpoint suits opinion scales where indifference is a real position. A "Don't know" or "Not applicable" option suits questions that some respondents cannot answer. Keep the two separate so you can tell them apart in analysis.

How many people do I need for a pilot survey?#

There is no fixed number. A small group from your real audience is usually enough to reveal confusing wording. Keep testing until a new round of pilot respondents stops finding problems.

Start building for free