Notes on Data Variance, Mismatches, and Curiosities

This page contains guidance on how to interpret changes over time in the data featured in our Global Digital Reports series.

It includes general notes, which are applicable to most of the charts and data points in our reports, and specific notes, which capture important changes and anomalies in individual datasets or data points.

You can find the specific notes further down the page. Please note that specific notes are listed in reverse chronological order, with the most recent updates towards the top of the page.

ADVERTISEMENT

General notes

The reports and analysis articles that we feature on DataReportal use data from a wide variety of sources, including data published by market research companies, internet and social media platforms, governments and public bodies, and many more. We also include extrapolations of this data, and our own proprietary analysis.

Wherever possible, we prioritise data sources that provide broader geographical coverage, in order to minimise potential variations between data points, and to offer more reliable comparison across countries. However, where we believe that individual, standalone metrics provide a more reliable reference, we use those numbers to provide more accurate reporting and more representative insights. Please note that some data points may only be available for a limited selection of countries or on an ad hoc basis, so we may not be able to report the same data for all geographies, or in each update to our reports.

From time to time, we may also change the source(s) that we use to calculate or inform specific data points. As a result, some figures may appear to change in unexpected ways between different reports in the Global Digital Reports series. Wherever we’re aware of these changes, we include details in the footnotes of each relevant chart. However, please use caution when comparing any data from different reports, because changes to research samples, base data, research methodologies, and how the source organisation reports its data may mean that values for what may appear to be the same data point are not directly comparable.

Furthermore, due to differing data collection and treatment methodologies, and the different periods during which data have been collected, there may be significant differences in the reported metrics for what appear to be similar data points throughout our reports. For example, data from surveys often varies over time, even if that data has been collected by the same organisation using the same approach in each wave of their research.

In particular, reports of internet user numbers may vary considerably between different sources and over time. In part, this is because there are significant challenges associated with collecting, analysing, and publishing internet user data on a regular basis, not least because research into internet adoption necessitates the use of face-to-face surveys. Different organisations may also adopt different approaches to sampling the population for research into internet use, and variations in factors such as the age range of the survey population, or the balance between urban and rural respondents, may play an important role in determining eventual findings.

Prior to our Digital 2021 reports, we included data sourced from social media platforms’ self-service advertising tools in our calculations of internet user numbers, but we no longer include this data in our internet user figures. This is because the user numbers reported by social media platforms are typically based on active user accounts, and may not represent unique individuals. For example, one person may maintain more than one active presence on the same social media platform (i.e. “duplicate” accounts). Similarly, some accounts may represent “non-human” entities, including: pets and animals; historical figures; businesses, interests and causes, groups and organisations; places of interest; etc.

As a result, the figures that we report for social media users may now exceed the figures that we report for internet users, and in some instances may even exceed figures for total population. However, such instances do not necessarily represent ‘errors’ in the data, or in our reporting. For example, these differences may indicate delays in the reporting of internet user numbers, or they may indicate higher instances of duplicate or “non-human” social media accounts. In situations where social media user numbers exceed figures for population, it may be worth considering that official population figures might not include migrant workers or temporary residents, but these individuals may still be counted in social media platforms’ reports of users for that location.

Similarly, the figures that we report for the advertising audience reach of certain social media platforms may be quite different to the total monthly active user figures we report for those same platforms. This difference may be caused by various factors, including:

  • The impact of economic sanctions, which may prevent companies from selling advertising in certain territories.

  • Individual user behaviours, which may result in some active users not being shown any advertising in the relevant reporting period. For example, if a Facebook user only checks a friend’s profile, they would still qualify as an active user, but they may not see any advertising, because adverts do not currently appear on people’s Facebook profile pages.

  • Company policies which may restrict the serving or targeting of advertising to specific users or user groups.

  • A lack of infrastructure enabling the sale of advertising in certain locations.

ADVERTISEMENT

Specific notes

January 2022 – internet users: Research into internet adoption relies heavily on face-to-face interviews, because even phone-based interviews can inflate adoption rates by focusing on people who already have access to technologies that enable internet connectivity. COVID-19 has made it more difficult to conduct these face-to-face interviews, resulting in a meaningful reduction in internet user research over the past two years. Research has started to return to normal, but because of these delays, new data may reflect changes over two or more years. As a result, figures for year-on-year growth in internet users in our Digital 2022 reports may represent growth over periods of more than one year, especially where updated figures were not available in time for the production of our Digital 2021 reports.

January 2022 – social media users: The figures we report for social media users rely heavily on advertising reach data published by social media platforms, and any changes in how those platforms report reach can have a significant impact on the numbers we report too. Various changes in platform reporting over recent months (see below) mean that some numbers in this year’s reports are not directly comparable with figures for the same data points published in previous reports, even though all figures were correct at the time of production. Where we’ve been able to calculate representative growth figures, we’ve included those figures in this year’s reports, but where we haven’t included figures for change over time, it’s likely that any comparisons to historical data will result in inaccuracies. As a result, please avoid comparing social media user figures in this year’s reports with figures included in our previous reports.

January 2022 – Meta: In Q4 2021, Meta made some important changes to how its self-service tools report the potential advertising reach of its platforms, including a move to publishing audience figures as a range, instead of as an absolute number. Our analysis suggests that Meta also revised its base data around this time, resulting in some important corrections to published audience figures for Facebook and Messenger. As a result, audience figures for Facebook and Messenger included in our reports published after 01 January 2022 are not comparable with similar data points included in our reports published before that date.

October 2021 – Twitter: In Q3 2021, Twitter made important changes to how its self-service tools report potential advertising reach. As a result, audience figures for Twitter included in our reports published after 01 October 2021 are not comparable with similar data points included in our reports published before that date.

January 2021 – internet users: starting 01 January 2021, we no longer include social media user numbers in the figures we publish for internet users, except where no other data are available. As a result, internet user numbers included in our reports published after 01 January 2021 may not be directly comparable with similar data points included in our reports published before that date.

ADVERTISEMENT

Further information

If you have any questions about specific data points used in our reports, please see our notes on data sources and methodologies.

And if you’d like to offer your organisation’s data for potential inclusion in our future reports, please click here to email us.