People often ask whether spending time on social media is good or bad for us. To answer this question, researchers need accurate ways to measure how much time people spend on platforms like Facebook, among other things. The most common approach, found in the vast majority of published studies, is through survey questions asking participants how much time they spent on these platforms. However, participants’ reports of their own use have well-documented limitations. Participants may not report accurately, because they either can’t recall or don’t know. Keeping track of time is hard, and people may report in biased or skewed ways. Some people may be more prone to recall errors. Further, validating these self-report measures is challenging in the absence of data from internal server logs.
Our aim is to provide researchers with validated self-report time measures that more closely capture people’s actual time spent on Facebook. In our latest paper, “How Well Do People Report Time Spent on Facebook? An Evaluation of Established Survey Questions with Recommendations” (CHI 2020), we evaluate common survey questions from the literature, provide recommendations to researchers, and provide translations for 14 languages.
We compared data from 10 self-reported Facebook-use survey measures deployed in 15 countries (N = 49,934) against data from Facebook’s server logs. We found that:
- Participants significantly overestimated how much time they spent on Facebook and underestimated the number of times they visited. For example, on one survey question, people overestimated how much time they spent on Facebook by an average of 3.2 hours per day (see Figure 1).
- Self-reported time spent was only moderately correlated with actual Facebook use (r = 0.23–0.42 across the 10 questions).
- Some questions caused underestimation, while others caused overestimation. Only 27 percent were able to respond accurately even on the best-performing question.
- The more time people spent on Facebook, the more likely it was that they misreported their time.
- Teens and young adults have more error reporting their time on Facebook, which is notable because of the high reliance on college-aged samples in many fields.
Figure 1
Informed by these results, we recommend the following to researchers aiming to measure time spent on Facebook:
1. To reduce measurement error, we recommend that researchers ask participants to report data from time management tools like Your Time on Facebook rather than try to estimate it themselves.
2. When time spent must be collected via self-report, we recommend the following wording from Ellison et al. (2007), which had the lowest error in our study.
- In the past week, on average, approximately how much time PER DAY have you spent actively using Facebook?
- Less than 10 minutes per day
- 10–30 minutes per day
- 31–60 minutes per day
- 1–2 hours per day
- 2–3 hours per day
- More than 3 hours per day
3. Because self-reports of time spent are imprecise, we suggest that researchers not use these values directly but rather interpret people’s self-reported time spent as a noisy estimate of where they fall on a distribution relative to other respondents.
While our focus here is “time spent” questions, because these are very common in the literature, a growing body of studies shows that merely examining the amount of time an individual uses social media is inadequate for many questions of interest (such as how social media use might be associated with loneliness, social comparison, or academic performance). Instead, we recommend focusing on how people use social media, as discussed in the following studies:
- The relationship between Facebook use and well-being depends on communication type and tie strength by Moira Burke and Robert E. Kraut
- Social capital and resource requests on Facebook by Nicole B. Ellison, Rebecca Gray, Cliff Lampe, and Andrew T. Fiore
- Do social network sites enhance or undermine subjective well-being? A critical review. by Philippe Verduyn, Oscar Ybarra, Maxime Résibois, John Jonides, and Ethan Kross
Beyond these implications to researchers, we hope tools such as Your Time on Facebook provide people with more insight into the time they spend on our platform, and foster conversations around their perceptions of use and online habits. Connecting our own perceptions of Facebook use (“How is the time I spend on Facebook good/bad for me?”) and what scientific research tells us about social media’s impact on our lives is also crucial. In that regard, we hope insights from our study provide readers with tools to critically engage with science communication on social media use and well-being.
With the evolution of the platform with new features and shifts in how people use Facebook, even the strongest survey measures are likely to evolve. That said, employing a stable set of established measures is an important methodological practice for researchers to support comparative work within the scientific community. We hope to make a positive contribution by providing such validated measurements that support international, academic, and comparative work on the impact of Facebook on people’s lives.
The post Recommendations for researchers to more accurately measure time spent on Facebook appeared first on Facebook Research.