How to write effective survey and research questions

Posted on 07/09/2022 by Annette Aiken.

The use of surveys to gather and triangulate evidence for scrutiny reviews is easier, now that we can be more confident that a majority of local people have access to the tools to fill them in online – making the use of hard-copy questionnaires unnecessary other than in situations where scrutiny is keen to gather insight from a group of people known to experience digital exclusion.

But surveys need to be well-designed to elicit useful results. Well-planned surveys can gather rich data about local people’s needs and views, which can complement information brought by council officers and other professionals. Poorly-planned, they can introduce confusion, or give rise to misleading findings. Even worse, poorly communicated surveys, published to the public by the scrutiny function, can give rise to the belief that the council is thinking about taking, or not taking, a particular course of action because of a confusion about what scrutiny is, and its role. Thinking about audience need and use is, therefore, a vital part of the art of crafting data capture.

Scrutiny practitioners should not let this put them off. In this blog, Annette Aiken, one of our Research and Project Officers, shares her insight into what the building blocks for a great survey look like. We hope that you can bear some of this advice in mind in thinking about how to integrate survey evidence into your scrutiny reviews.

Survey research is often used by researchers who wish to explain trends or features of large groups. It may also be used to assist those planning some more focused, in-depth study.^[1]

Having accurate data, obtained from local government surveys aimed at the public and other groups, will help councils to learn more about their resident demographic and better support the design and delivery of services. Surveys can also be used to obtain the views and perspectives of other populations. Examples could be staff/employees and other groups, such as general practitioners. Surveying different organisations, for instance town and parish councils. Additionally, data can be used to understand the needs of different people who live and/or work in the local area.

Much time and resources can be spent creating survey questions. Whether they be for quantitative or qualitative surveys, creating good survey questions goes a long way to increasing response rates to enable accurate, usable data driven insights to be gained and to promote robustness and reliability of results or findings. Surveys also engage the public and allow their voices to be heard in decision-making processes.

Obtaining good survey response rates and results can also be crucial for local authorities to obtain robust data, which is an essential basis for decision making, especially if regional or national benchmarking is to take place.^[2]

Using surveys as a method of consultation, to find out what residents and other stakeholders think about specific issues are constantly employed by councils, often as a statutory requirement and also in a non-statutory capacity.^[3]

Good survey questions should be well-worded, straightforward, easy to understand, clear, and to interpret by respondents in the same way, thus minimising bias. Poor questions can lead respondents, skew results and impact decisions that might be made as a result of survey findings.

A good survey question is also one that is asked at just the right moment, so that it’s relevant and elicits an accurate and honest response.^[4]

Solid preparation is required when creating surveys. In order to avoid common pitfalls, such as, bias, confusion or empty responses and to promote better quality responses as well as higher numbers of responses.

Consideration should be given to:

Who to consult- Will the general population be being consulted or are specific groups being targeted? Targeted groups might include representative groups, special interest groups, specific service users, faith groups. Different demographics should be considered. For example, older people, young people, homeless people or those identifying as LGBTQ+. Always keep in mind the audience the survey is targeted towards.
The type of insights that need to be gained from surveys should also be considered. Are general insights required or more specific insights? Giving some thought to how the research will be used is useful as this often helps to shape the type and number of questions that are required in a survey. This about what type of data is required to be collected. If insights into experiences and/ or characteristics are required, then a qualitative approach would be ideal. If numerical data is required, then a quantitative approach is required. There might be need for a combination of both approaches.
Method of consultation- Are the questions going to be qualitative or quantitative in nature or a mixture of both? Will the surveys be self-completion or not? How will the survey be distributed (by post, email, online, social media, text, consultation days, exhibitions, via an App).
Type of survey questions- There are many different types of survey questions which include multiple choice, open-ended, closed-ended, rating sale (ordinal), Likert scale, Yes/No, Nominal and demographic survey questions.^[5] All these different question types fulfil different purposes. Decide which question types to include in the survey and how many questions to include in the survey. Aim for a good mix of question types, including open-ended questions, which will gain deeper insights and will be more likely to keep respondents engaged and stimulated.

Top Tips to help write effective survey questions:

The top tips below form a step-by-step guide for all scrutineers to follow:

Explain the purpose of the survey so that potential respondents can see the value in their responses. For instance, their responses will be used to inform how services are improved/delivered/ experienced.
There are usually 4 components to a survey- the invitation, introduction, questions and close (including thanks).
Indicate the amount of time respondents might need to complete the survey.
Use clear, concise, plain language to make it easy to understand.
Be precise, definite, and forthright. Avoid ambiguous language.
Avoid using technical language, acronyms and jargon to minimise survey bias.
Keep a consistent tone in line with your organisation.
Include any further information that respondents might need.
Always include ‘Other’ and ‘prefer not to answer’ options. Respondents might want to provide an alternative answer, or not want to answer a question at all. Providing these responses within questions options provides respondents with flexibility.
Keep question phrasing neutral. Leave different options to account for variability in your survey respondents.
Avoid using double negatives.
Be aware of language that leans either positively or negatively.
Make sure that questions are grammatically correct. Avoid complex statements.
Avoid double-barred questions (two questions in one).
Encourage respondents to answer the questions by providing a response such as ‘Not sure’ and ‘Not applicable’.
If possible, keep the whole survey short. People do not want to spend a lot of time completing surveys. Having a short, concise survey will encourage completion rates and illicit more meaningful data
Use the ‘funnelling technique’. Start with broad questions, general questions that are easy to answer and encourage engagement. Place harder, more specific questions in the middle. Finish with easier questions.
Avoid bias. Biased questions are created in such a way as to favour one answer over the other answers. This will result in misleading data being gathered. Bias questions include leading questions, loaded questions and absolute questions (yes or no answer only) and using particular adjectives and adverbs in questions.
Test surveys with colleagues to ensure that questions can be understood, iron out and glitches and to catch any typos etc
It is recommended that surveys contain a maximum of between 10-15 questions.^[6]

Writing good survey questions takes time. Consideration should be given to several factors, such as who to consult, the type of insights that the survey hopes to elicit, the method of consultation and the types of survey questions to employ.

There are numerous tips which can be used to ensure that surveys are as effective as possible. Ensuring effective survey questions encourages higher response rates, promotes meaningful authentic insights that can be more accurate, reliable and unbiased.

Surveys engage the public and allow their voices to be heard in decision-making processes within their local area and beyond. Obtaining as many responses as possible ensures that the insights gained are from as wide a population as possible and that these insights are as robust and reliable as possible.

Useful Resources: