9 These figures only include cases issued in Year 9 of the survey. Since the completion date might vary significantly from when the sample was issued, the figures in this table do not match the number of online completes in the final dataset.
Break-offs
A break-off occurs when a participant enters the online questionnaire but does not complete it. Software allows this abandoned survey data to be captured.
These data can be analysed and used to identify problems with the survey, formatting issues on devices (which can arise on an ad-hoc basis due to device updates), indicate questions that respondents find difficult to answer or where there may be technical issues.
It is possible to quantify an overall break-off rate by dividing the number who abandoned the survey by the number who started the questionnaire. It should be noted that cases which had a break-off near the end of the online questionnaire could be included in the dataset.
Once the question on number of adults in the household had been answered in the online questionnaire, the case was treated as being productive.
The overall break-off rate among entrants to the online survey was 12.1%. This is relatively low, but represents an increase from previous years.
This was not seen to be a concern, however, since the introduction of QR codes increased the number of people accessing the online survey. This highlights the effectiveness of the design and format of the device-agnostic questionnaire.
-
Break-offs by section
The survey was split into the following seven broad groups for the break-off analysis:
Read more about Break-offs by sectionPre-survey
Any pages which you see before the survey begins.
Art & Culture
Questions relating to whether someone had taken part in a particular art or cultural activity, including when they have done it over the course of the year and how often.
Sport, Fitness & Recreation
Questions relating to activities that the participant has taken part in within the last year. This includes when during the year, they had undertaken them; participants are also asked about the frequency, duration and intensity of the activity, and if they have done it within the last 28 days.
Engagement
These questions relate to a respondent’s habits, motivations, opportunities and volunteering in sport or exercise.
Community & Life
This includes questions on a participant’s wellbeing, their goals and their community.
About you
This includes questions relating to the respondent’s health, as well as questions to ascertain the respondent’s demographic information.
Final questions
Questions based around re-contact, and participants are asked if they wish to receive the incentive or not.
Break-off by section
Section Dropout percentage 1 Pre-Survey 2.8% 2 Arts & Culture 6.0% 3 Sport, Fitness & Recreation 48.7% 4 Engagement 14.1% 5 Community & Life 4.1% 6 About you 18.1% 7 Final questions 6.2% Consistent with previous years of the survey, most breakoffs occurred in the ‘Sport, Fitness and Recreation’ section.
This includes questions on which activities respondents have taken part in within the last 12 months, with a range of follow-up questions depending on when they last took part in the activity.
At this stage the questionnaire becomes more complex with a range of drop-down menus, grids and questions, which require higher levels of cognition. Thus, it is perhaps of no surprise that respondents are more likely to break off here rather than at other points in the questionnaire.
In the Sport, Fitness and Recreation section, the highest break-offs were at questions asking about how often, for how long and how intense the activity was in the past 28 days (ACTIV3). This was also the case in Year 1 for both phase 1 and phase 2 of the questionnaire.
Showing the proportion of all break-offs that are ACTIV3, ACTYR or ACTIV1
Question Proportion ACTIV3 (how often, for how long and how intense) 28.9% ACTYR (which periods during the year) 5.9% ACTIV1 (which activities done in the last year) 4.4% ACTYR had the second highest break-offs at 5.9%. Break-offs may be due to respondent difficulty in recalling when they did the activity in the last year.
During cognitive interviews conducted in Year 1, participants struggled with this question and found it difficult to understand – instead they wanted to report when they last did an activity and did not expect to be asked all the periods in which they did it.
ACTIV1 is the question with the lowest break-off rate at 4.4%. This question presented respondents with a list of sports, recreational activities and exercises that they may have undertaken, in the past year.
This was presented as a list of types of activities. Each header can be clicked on and beneath it is a list of activities the respondent can select, or alternatively they can search for an activity.
Read less about Break-offs by section -
Break-off by device
Paradata is collected on a range of different metrics including browser, operating systems, device type and the brand of the device (where applicable).
Read more about Break-off by deviceWith a device-agnostic survey, it is important to look at not only the break-offs by section (and question within that) but by type of device as well. Collecting paradata allows us to do this.
Comparison of the proportion of completed interviews by type of device against the proportion of breaks-offs gives an insight into the difficulties that may be faced with the survey on particular devices.
The break-off rate remains higher on mobiles than on desktop/laptop computers or on tablets, as in previous years. Callegaro et al. (2015) have suggested that break-off rates tend to be two or three times higher for smartphone respondents compared to desktop/laptop respondents.
Completion status of the questionnaire by type of device
Read less about Break-off by deviceDevice Completes
n9Completes
%Break-offs
nBreak-off proportion Break-off rate Desktop / laptop 40,127 31.9% 3,824 22% 8.7% Smartphone 56,848 45.1% 9,627 55.3% 14.5% Tablet 3,088 2.5% 333 1.9% 9.7% Device unknown / other 25,888 20.6% 3,633 20.9% 12.3% -
Profile of achieved sample
The following table shows the profile of the achieved sample broken down by mode. Note that this is the profile for responses received between 16 November 2023 to 15 November 2024.
Read more about Profile of achieved sampleProfile of the achieved sample (unweighted) by mode
Demographic CAWI (Online) Paper Total Base 130,692 40,976 171,668 Gender Male 44.6% 43.7% 44.4% Female 54.7% 56.1% 55.0% Other 0.7% 0.1% 0.6% Age 16-24 7.7% 3.0% 6.6% 25-34 16.1% 5.3% 13.5% 35-44 18.6% 8.3% 16.2% 45-54 16.7% 11.1% 15.4% 55-64 18.0% 21.1% 18.7% 65-74 15.0% 26.2% 17.7% 75-84 6.7% 19.8% 9.8% 85+ 1.1% 5.1% 2.0% NSSEC Managerial, administrative and professional occupations
(NS-SEC 1-2)68.2% 59.4% 6.4% Intermediate occupations
(NS-SEC 3)5.9% 8.0% 6.3% Small employers and own account workers
(NS-SEC 4)6.3% 8.3% 6.7% Lower supervisory and technical occupations
(NS-SEC 5)9.5% 11.9% 10.0% Semi-routine and routine occupations
(NS-SEC 6-7)2.7% 2.7% 2.7% Long-term unemployed or never worked
(NS-SEC 8)4.0% 1.9% 3.6% Full-time student and other/unclassified
(NS-SEC 9)3.4% 7.9% 4.3% Working status Working full or part time 60.6% 42.2% 56.3% Unemployed 3.6% 2.2% 3.2% Not working (e.g. retired, looking after children) 28.6% 51.9% 34.1% Student full or part time 4.2% 1.6% 3.6% Other working status 3.0% 2.1% 2.8% Ethnicity White British 78.4% 87.5% 80.6% White Other 7.1% 4.4% 6.5% Asian 7.6% 3.9% 6.7% Black 2.7% 2.0% 2.5% Chinese 1.1% 0.7% 1.0% Mixed 2.0% 0.9% 1.7% Other 1.1% 0.7% 1.0% The profile of the achieved sample varies from the overall profile of the population. There are several factors that contribute to this:
Survey topic: It has been shown in recent literature that the profile of push-to-web survey respondents varies depending on survey topics; this is likely to have had some impact on the profile of responders. Non-responders may not have a significant interest in the areas that the survey covers and so will not be inclined to complete the questionnaire.
Modern changes in household structure: It has been shown that there is a growing proportion of young adults (between the age of 20-34) that live at home with their parents. These are unlikely to be the individual in the households who opens the mail (as the letter is sent to the household as a whole, and not an individual) and hence has an impact as to who actually completes the survey.
Online population: While the proportion of those online in England is overall very high, there is a significant proportion of certain sub-groups who are not online. They will not be able to complete the survey online and survey fatigue may have set in by the time the second reminder arrives (with the paper questionnaire). This may have some, if only a small, impact on the overall profile and the response rate.
Throughout the life of the Active Lives study, there has been a focus on improving the profile of the overall achieved sample and increasing return rates. This will continue to be a priority.
Read less about Profile of achieved sample