Over the last two decades, telephone surveys based on the random generation of telephone numbers, whether landline or mobile, have gained momentum both in France and internationally. This method of random dialling allows interviewers to reach households and individuals not included in telephone directories. In parallel, the proportion of the population with a mobile phone has increased, and the use of landline phones is decreasing. The ease with which people now communicate via mobile phones makes it possible to consider a survey strategy based solely on mobile phone numbers. For Santé publique France, the question is particularly relevant to the Health Barometer survey. These repeated surveys of the general population, carried out since 1992 by Santé publique France* , are valuable sources of information for monitoring changes in French health behaviours (see Box - The Santé publique France Health Barometer. A major challenge for these surveys is adapting to maintain good quality, while preserving the monitoring of trends that is central to the expected results. For this reason, any change in the survey method must first involve an impact assessment.
What would be the impact of moving to a Health Barometer conducted 100% via mobile phones in terms of survey productivity, respondent characteristics and health behaviour estimates? The article just published in the research section of the journal BMC Medical Research Methodology  provides some answers.
3 questions for: Noémie Soullier, Data Sciences Division (DATA), survey unit, Santé publique France
Surveys based on random digit dialling, such as the Santé publique France Health Barometer, make it possible to reach almost the entire French population, since more than 99% is now equipped with a landline or mobile telephone (ICT Survey 2021, INSEE) (1). These surveys are fairly simple to set up as they involve randomly dialling telephone numbers. Over time, they have had to adapt to changes in telephone equipment and usage, particularly with the emergence and development of mobile phones. In the early 1990s, for example, telephone numbers were drawn at random from the phone book; then landline numbers were dialled at random, with incrementing to include numbers that didn’t appear in directories. Mobile phones have gradually integrated this type of survey design and now represent more than half of the numbers called (60% to 70%). The question asked in this study is: " Can we do without calling landline numbers and go 100% mobile ? ” when conducting a general population survey in metropolitan France. In order to answer, we assessed the potential impact of this change on survey results, both in terms of population characteristics and of the estimates for key health indicators (perceived health status, sedentary lifestyle, smoking, suicidal behaviour). The analysis is based on data from the Santé publique France 2017 Health Barometer : we compared the results obtained from the whole sample (≈25,000 people) constituted by a mix of landline and mobile numbers (the method applied for the Health Barometer since 2014) to the results that would be obtained using only the data collected by mobile phone (≈15,000 people). This impact assessment of a change in survey methodology is all the more important for repeated surveys such as the Health Barometer, which study trends, and for which a break in the series could be detrimental to interpretations.
Interviewing on mobile phones has several advantages. Firstly, mobile phones are in most cases a personal device: interviewing the main user of the number dialled is therefore most of the time the same as interviewing the person who answers the phone. By contrast, a landline telephone number corresponds to a household and a single respondent must be drawn for interview from within that household. Calling a mobile number therefore reduces the duration of the interview by avoiding this selection phase (about 2 minutes) and also increases the precision of the estimates by employing a single-stage sampling design (as opposed to two-stage design landline phones). In addition, the number of calls required to interview a person is lower on a mobile phone, allowing the survey field to be covered more quickly: on average, it is necessary to randomly draw and call 6 mobile phone numbers to obtain an interview, compared to 16 numbers on a landline phone. This significant difference is mainly due to a higher proportion of ineligible numbers (out of use, companies...) among landline phones. Finally, on a mobile, the person chooses where to pick up the phone and where to continue the interview, which can create situation of confidentiality and favour sincere answers.
The disadvantage of mobile phone numbers is that very few (1%) are found in directories, making it impossible to send people a letter to notify them that the interviewer will call. This upstream information stage is systematically carried out for telephone numbers registered in directories and helps to encourage participation. However, this limitation is not specific to mobile phones, as less than 10% of landline numbers are found in the phone book. Another disadvantage of calling on a mobile phone is that the number is displayed and the person may decide not to answer if they do not recognize it. In 2021, 68% of people with a mobile phone said that they only answer when they know the number calling; this is the case for 48% of landline phone owners who systematically screen calls (ICT Survey 2021, INSEE) (1).
The paper shows that all parts of the population can be reached through a survey conducted using only mobile phone numbers and that the distribution of the respondent sample would be as close to the population structure as with a survey conducted using 60% mobile/40% landline, which was the case with the Santé publique France 2017 Health Barometer. However, there are a few differences to be noted: using 100% mobile facilitates reaching populations who are younger (under 35 years old), educated (to Baccalaureate level or over) and urban, while conversely, older people (65-75 years old), people with fewer qualifications and people living in rural areas are more difficult to reach.
Estimates of health behaviours are similar between the two types of survey (100% mobile vs 60% mobile/40% landline). For example, the proportion of people who say they are limited in their daily activities is estimated at 21.6% in 60% mobile/40% landline surveys, and 21.1% in 100% mobile surveys; the proportion of sedentary people is estimated at 8.7% and 8.9%, respectively. Thus, moving to a 100% mobile survey would not radically change the associated health messages. There are, however, some reservations: the estimate of daily smokers is higher in the 100% mobile survey (+0.6 points for the population as a whole, +1.0 point among 18-30 year olds and +1.1 points among 60-75 year olds). The magnitude of the difference remains measured, but nevertheless corresponds to the evolution that can be observed from one year to the next for this indicator. In other words, this difference could mask a favourable or unfavourable evolution in terms of health behaviour.
The results of our study allow us to consider a 100% mobile random dialling design for future surveys conducted by the agency: we have shown that this method provides a respondent sample with a structure close to that of the population and similar estimates of health behaviours with greater precision. It is a more productive solution that could be favoured for new surveys, especially those aimed at a young public. Regardless of the equipment used to make the call, an important factor in this type of survey is the call protocol, which must include high insistence (many calls at various time slots), which in turn would make it possible to quantify the type of respondents who are difficult to reach or less inclined to respond. These people – for example those who agree to participate after hanging up the first time before the interviewer could introduce the survey – may be less aware of health messages. For example they have less often heard of the “Mois sans Tabac” [“Smoke-Free Month”] campaign, as shown by the results of the Santé publique France 2019 Health Barometer (2). This is why it is important to combine the quality of the sampling and of the protocol: it is what ensures the quality of the survey and the estimates, and is the way to achieve a good representation of all situations.
Santé publique France Health Barometer surveys
The primary objective of the Santé publique France Health Barometer is to gain better knowledge and understanding of the health attitudes and behaviours among people living in France in order to build legitimate and effective interventions. This survey does not seek to measure the health status of the population as such, but in fact the health perceptions and practices studied partly determine this.
For the past 30 years, these repeated surveys have aimed to monitor the main behaviours, attitudes and perceptions related to risk-taking and health status within the population residing in France: smoking, alcohol consumption, illicit drug consumption, vaccination practices, sexuality, cancer screening, physical activity, nutrition, quality of life, sleep, accidents, mental health, etc.
They are based on random samples of landline and mobile phone numbers and an extensive call protocol designed to maximise participation and thus represent the full range of situations in the population. The data is collected via a questionnaire administered by telephone interviewers using the Computer Assisted Telephone Interviewing system. Each survey provides a 'snapshot' of a given health issue at a given time. Over two decades, the method used has constantly evolved to adapt to the technical and administrative constraints imposed by the end of the national telecommunications monopoly, and then by the diversification of telephone equipment and uses.
*Carried out from 1992 to 2001 by the French Health Education Committee (CFES), then by the National Institute of Health Education and Prevention (INPES) from 2002 to 2016, the year Santé publique France was founded.
 Noémie Soullier, Stéphane Legleye, and Jean-Baptiste Richard. Moving towards a single-frame cell phone design in random digit dialing surveys: considerations from a French general population health survey BMC Medical Research Methodology (2022) 22:94
Other references cited :
(1) The use of information and communication technologies by households between 2009 and 2021. ICT household surveys – INSEE Results. Enquêtes sur les TIC auprès des ménages - Insee Résultats. INSEE FOCUS. No 259. Paru le : 24/01/2022.
(2) Soullier N, Richard JB, Gautier A. Santé publique France Health Barometer 2019. MethodSaint‑Maurice : Santé publique France, 2021 : 14 p