Facebook research at KDD 2020

by Facebook AI

Facebook researchers and engineers in data science, data mining, knowledge discovery, large-scale data analytics, and more will be presenting their research at the Conference on Knowledge Discovery and Data Mining (KDD). Researchers will also be giving talks and participating in workshops and tutorials throughout the conference.

The Facebook Core Data Science team is presenting two research papers: “CLARA: Confidence of labels and raters” and “TIES: Temporal interaction embeddings for enhancing social media integrity at facebook.” CLARA, which is described more in this blog post, is a system built and deployed at Facebook to estimate the uncertainty in human-generated decisions in our content review process. TIES is a deep learning, application-agnostic, scalable framework that leverages interactions between accounts to improve the safety and security of our online community.

For more information on Facebook’s presence at KDD this year, from August 23 to 28, check out the Facebook at KDD page.

Facebook research being presented at KDD 2020

AutoCTR: Towards automated neural architecture discovering for click-through rate prediction
Qingquan Song, Dehua Cheng, Eric Zhou, Jiyan Yang, Yuandong Tian, Xia Hu

Click-through rate (CTR) prediction is one of the most important machine learning tasks in recommender systems, driving personalized experience for billions of consumers. Neural architecture search (NAS), as an emerging field, has demonstrated its capabilities in discovering powerful neural network architectures, which motivates us to explore its potential for CTR predictions. Due to (1) diverse unstructured feature interactions, (2) heterogeneous feature space, and (3) high data volume and intrinsic data randomness, it is challenging to construct, search, and compare different architectures effectively for recommendation models. To address these challenges, we propose an automated interaction architecture discovering framework for CTR prediction named AutoCTR. Via modularizing simple yet representative interactions as virtual building blocks and wiring them into a space of direct acyclic graphs, AutoCTR performs evolutionary architecture exploration with learning-to-rank guidance at the architecture level and achieves acceleration using low-fidelity model. Empirical analysis demonstrates the effectiveness of AutoCTR on different data sets comparing to human-crafted architectures. The discovered architecture also enjoys generalizability and transferability among different data sets.

CLARA: Confidence of labels and raters
Viet-An Nguyen, Peibei Shi, Jagdish Ramakrishnan, Udi Weinsberg, Henry C. Lin, Steve Metz, Neil Chandra, Jane Jing, Dimitris Kalimeris

Large online services employ thousands of people to label content for applications such as video understanding, natural language processing, and content policy enforcement. While labelers typically reach their decisions by following a well-defined protocol, humans may still make mistakes. A common countermeasure is to have multiple people review the same content; however, this process is often time-intensive and requires accurate aggregation of potentially noisy decisions.

In this paper, we present CLARA (confidence of labels and raters), a system developed and deployed at Facebook for aggregating reviewer decisions and estimating their uncertainty. We perform extensive validations and describe the deployment of CLARA for measuring the base rate of policy violations, quantifying reviewers’ performance, and improving their efficiency. In our experiments, we found that CLARA (a) provides an unbiased estimator of violation rates that is robust to changes in reviewer quality, with accurate confidence intervals, (b) provides an accurate assessment of reviewers’ performance, and (c) improves efficiency by reducing the number of reviews based on the review certainty, and enables the operational selection of a threshold on the cost/accuracy efficiency frontier.

Compositional embeddings using complementary partitions for memory-efficient recommendation models
Hao-Jun Michael Shi, Dheevatsa Mudigere, Maxim Naumov, Jiyan Yang

Modern deep learning-based recommendation systems exploit hundreds to thousands of different categorical features, each with millions of different categories ranging from clicks to posts. To respect the natural diversity within the categorical data, embeddings map each category to a unique dense representation within an embedded space. Since each categorical feature could take on as many as tens of millions of different possible categories, the embedding tables form the primary memory bottleneck during both training and inference. We propose a novel approach for reducing the embedding size in an end-to-end fashion by exploiting complementary partitions of the category set to produce a unique embedding vector for each category without explicit definition. By storing multiple smaller embedding tables based on each complementary partition and combining embeddings from each table, we define a unique embedding for each category at smaller cost. This approach may be interpreted as using a specific fixed codebook to ensure uniqueness of each category’s representation. Our experimental results demonstrate the effectiveness of our approach over the hashing trick for reducing the size of the embedding tables in terms of model loss and accuracy, while retaining a similar reduction in the number of parameters.

Embedding-based retrieval in Facebook search
Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, Linjun Yang

Search in social networks such as Facebook poses different challenges than in classical web search: Besides the query text, it is important to take into account the searcher’s context to provide relevant results. Their social graph is an integral part of this context and is a unique aspect of Facebook search. While embedding-based retrieval (EBR) has been applied in web search engines for years, Facebook search was still mainly based on a Boolean matching model. In this paper, we discuss the techniques for applying EBR to a Facebook Search system. We introduce the unified embedding framework developed to model semantic embeddings for personalized search, and the system to serve embedding-based retrieval in a typical search system based on an inverted index. We discuss various tricks and experiences on end-to-end optimization of the whole system, including ANN parameter tuning and full-stack optimization. Finally, we present our progress on two selected advanced topics about modeling. We evaluated EBR on verticals for Facebook Search with significant metrics gains observed in online A/B experiments. We believe this paper will provide useful insights and experiences to help people on developing embedding-based retrieval systems in search engines.

ShopNet: Unified computer vision model trunk and embeddings for commerce
Sean Bell, Yiqun Liu, Sami Alsheikh, Yina Tang, Ed Pizzi, Michael Henning, Karun Singh, Fedor Borisyuk

In this paper, we present ShopNet, a deployed image recognition system for commerce applications. ShopNet leverages a multi-task learning approach to train a single computer vision trunk. We achieve a 2.1x improvement in exact product match accuracy when compared to the previous state-of-the-art Facebook product recognition system. We share our experience of combining different data sources with wide-ranging label semantics and image statistics, including learning from human annotations, user-generated tags, and noisy search engine interaction data. We experiment with a diverse set of loss functions, optimizing jointly for exact product recognition accuracy and various classification tasks. We provide insights on what worked best in practice. ShopNet is deployed in production applications with gains and operates at Facebook scale.

TIES: Temporal interaction embeddings for enhancing social media integrity at Facebook
Nima Noorshams, Saurabh Verma, Aude Hofleitner

Since its inception, Facebook has become an integral part of the online social community. People rely on Facebook to make connections with others and build communities. As a result, it is paramount to protect the integrity of such a rapidly growing network in a fast and scalable manner. In this paper, we present our efforts to protect various social media entities at Facebook from people who try to abuse our platform. We present a novel temporal interaction embeddings (TIES) model that is designed to capture rogue social interactions and flag them for further suitable actions. TIES is a supervised, deep learning, production-ready model at Facebook-scale networks. Prior works on integrity problems are mostly focused on capturing either only static or certain dynamic features of social entities. In contrast, TIES can capture both of these variant behaviors in a unified model owing to the recent strides made in the domains of graph embedding and deep sequential pattern learning. To show the real-world impact of TIES, we present a few applications especially for preventing spread of misinformation, fake account detection, and reducing ads payment risks in order to enhance the Facebook platform’s integrity.

Training deep learning recommendation model with quantized collective communications
Jie (Amy) Yang, Jongsoo Park, Srinivas Sridharan, Ping Tak Peter Tang

Deep learning recommendation model (DLRM) captures our representative model architectures developed for click-through rate (CTR) prediction based on high-dimensional sparse categorical data. Collective communications can account for a significant fraction of time in synchronous training of DLRM at scale. In this work, we explore using fine-grain integer quantization to reduce the communication volume of alltoall and allreduce collectives. We emulate quantized alltoall and allreduce, the latter using ring or recursive-doubling and each with optional carried-forward error compensation. We benchmark accuracy loss of quantized alltoall and allreduce with a representative DLRM model and Kaggle 7D data set. We show that alltoall forward and backward passes and dense allreduce can be quantized to 4 bits without accuracy loss, compared to full-precision training.

Other activities at KDD 2020

Applied data science invited talks

Data paucity and low resource scenarios: Challenges and opportunities
Mona Diab, invited speaker

Preserving integrity in online social media
Alon Halevy, invited speaker

Hands-on tutorials

Building recommender systems with PyTorch
Dheevatsa Mudigere, Maxim Naumov, Joe Spisak, and Geeta Chauhan, presenters

Special days

Deep learning day
Joelle Pineau

Workshops

Humanitarian mapping workshop
Shankar Iyer, initiative lead/chair
Kristen Altenburger, Eugenia Giraudy, Alex Dow, Paige Maas, Alex Pompe, Eric Sodomka, program committee

Workshop on applied data science for Healthcare
Paper: Information extraction of clinical trial eligibility criteria
Yitong Tseo, M. I. Salkola, Ahmed Mohamed, Anuj Kumar, Freddy Abnousi

Workshop on deep learning practice for high-dimensional spare data
Liang Xiong, workshop chair

Workshop on mining and learning with graphs
Aude Hofleitner, organizer
Jin Kyu Kim, program committee

The post Facebook research at KDD 2020 appeared first on Facebook Research.

Read More

Improving the accuracy of Community Standards enforcement by certainty estimation of human decisions

Improving the accuracy of Community Standards enforcement by certainty estimation of human decisions

by Facebook AI

What we did

Online services have made great strides in leveraging machine-learned models to fight abuse at scale. For example, 99.5 percent of takedowns on fake Facebook accounts are proactively detected before users report them. However, despite this progress, there are many areas where large-scale systems still rely on human decisions for a range of tasks, including collecting labels for training models, enforcing a range of policies, and reviewing appeals.

An obvious challenge that arises when relying on human reviewers is that we are inherently noisy and potentially biased decision-makers. While bias is a trait of the individual, noise can result from subjectivity or ambiguity of the decision guidelines, or from simple mistakes that are commonly the result of fatigue or pressure. In this work, we consider three applications where mistakes made by human decisions can have negative outcomes:

  • Enforcement: When community standards are being enforced, an incorrect decision can result in taking down a benign piece of content from the platform or leaving violating content on the platform.
  • Training machine learning models: Using inaccurate human-generated “ground truth” labels might lead to inaccurate models.
  • Prevalence estimation: Prevalence is the percentage of policy-violating content out of all content seen by Facebook users. It is computed by sampling content and sending it to reviewers, who review it for violations. Failing to consider mistakes in these reviews can lead to incorrect prevalence estimates and confidence intervals.

A scalable solution to reduce the likelihood of mistakes is to assign multiple reviewers to each task. If available, it is possible to augment human decisions with additional nonhuman signals, such as scores from machine learning models. A key challenge that rises in these settings is the need to aggregate multiple potentially conflicting decisions and provide an estimate for the certainty of the decision.

In our paper, to be published at the 2020 ACM Conference on Knowledge Discovery and Data Mining, we present CLARA (Confidence of Labels and Raters), a system built and deployed at Facebook to estimate the uncertainty in human-generated decisions. We show how CLARA is used at Facebook to obtain more accurate decisions overall while reducing operational resource use.

How we did it

We follow a rich body of research on crowdsourcing and take a Bayesian probabilistic approach to define different latent variables and the generative process of the observed data. In particular, the observed data includes a set of items, each of which receives multiple labels and potentially one or more scores from machine learning models. CLARA estimates the following latent variables:

  • Overall prevalence: The rate at which each label category occurs
  • Per-reviewer confusion matrix: Each reviewer’s ability to correctly label items of different true label categories
  • Per-item true label: The true latent label category of each item
  • Score mixture: The different score distributions of items from different label categories

For posterior inference, we implemented a collapsed Gibbs sampling algorithm to infer the values of all latent variables given the observed data. Figure 1 shows the graphical model of CLARA together with illustrative examples of the observed and latent variables.

Figure 1

Results

We’ve deployed CLARA at scale at Facebook. While similar underlying models have been studied in the literature, this work provides the details of a large-scale, real-world deployment of a complete system, with both offline and online aggregation and uncertainty estimation capabilities.

Figure 2 illustrates an overview of how CLARA is deployed at scale in production at Facebook.

Figure 2

One of the key applications where we use CLARA in Facebook is the efficient allocation of labeling resources based on confidence scores. We achieve this by obtaining additional reviews only when the decision confidence given by CLARA is not sufficiently high. This results in a cost/accuracy trade-off, where higher levels of decision confidence result in additional reviews. An example trade-off curve, which uses simulated “ground truth” and labeling mistakes, is shown in Figure 3. The figure depicts the change in accuracy (left) and mean absolute error (right) as a function of the percent of labels. Compared to a random sampling baseline, the figure shows that CLARA provides a better trade-off curve, enabling an efficient usage of labeling resources. In a production deployment, we found that CLARA can save up to 20 percent of total reviews compared to majority vote. You can find more details and results in our paper.

Figure 3

How we are extending this work

The current implementation of CLARA leverages machine learning scores by treating them as nonbinary “artificial reviewers.” However, we observe that human mistakes are often correlated with the difficulty of the task, which can be reflected in the machine learning score. We are developing a continuous “confusion function” and prevalence function, which takes into account the difficulty of the task as captured by the machine learning score.

The post Improving the accuracy of Community Standards enforcement by certainty estimation of human decisions appeared first on Facebook Research.

Read More

Fellowship 101: Facebook Fellow Daricia Wilkinson outlines the basics for PhDs

by Facebook AI

The Facebook Fellowship Program supports talented PhD students engaged in innovative research in any year of their PhD study. Applications for the 2021 Fellowship cohort recently opened on August 10, and they will close on October 1.

Apply
“Each year, the program gets more and more competitive. Last year, we received around 1,875 applications — double the amount as the year before,” says Sharon Ayalde, Program Manager for the Facebook Fellowship Program. “We’re looking forward to the high quality of applications that we see every year.”

To prepare for this year’s Fellowship applications, we connected with Daricia Wilkinson, 2019 Fellow in UX/Instagram, to discuss the fellowship basics. Wilkinson is a PhD student in the Human-Centered Computing program at Clemson University, advised by Dr. Bart Knijnenburg. Her research interests are at the intersection of people and technology, and she is passionate about solving problems from a user-centered perspective.

Inspired by Wilkinson’s Medium post about how to make a successful PhD fellowship, this Q&A outlines the most common questions Wilkinson receives about fellowships, research statements, and the application process.

Q: How do fellowships work?

Daricia Wilkinson: If you are an incoming PhD student, you will learn about assistantships from your university (either teaching or research assistantship) that support your tuition and stipend. In contrast, fellowships are a source of external funding that could be offered by a governmental organization or a company. However, not all fellowships are the same. There are some key differences that can help guide you when deciding which fellowships to apply to:

  • Fellowship amount: You will probably recognize this fairly early, but the amount being offered could vary significantly. The typical range for PhD fellowships is $10,000 to $40,000.
  • Type of support: The support given could contribute toward covering tuition, your stipend, or travel. Some fellowships may only offer one type of support. It should also be noted that the money is sometimes paid directly to the school and not to you. That might make it easier or more difficult depending on your situation.
  • Duration: You may be offered the amount in a one-time payment, or it may be offered over a set number of years.
  • Additional offers: Some fellowship programs are more robust and hands-on than others. It is quite possible to be offered the opportunity to interview for internships. Programs that could possibly lead to your being hired want you to know this. I’d recommend taking some time to comb through each program’s FAQ to see whether this is an option. Beyond internships, some organizations allow you to collaborate and network with their research teams, which could be an invaluable experience.

Q: How is the Facebook Fellowship different from others?

DW: First, the Facebook Fellowship is very prestigious. Unlike many other fellowship opportunities, the Facebook Fellowship offers a very generous level of support. Facebook pays your tuition, and you are provided with a very competitive stipend of $42,000, meant for living costs and travel support.

Second, the Fellowship offers incredibly valuable networking opportunities. The Facebook Fellowship Summit, hosted virtually this year, is one of these opportunities. At the summit, Fellows are invited to a paid trip to Facebook headquarters in Menlo Park, where they can present their research and meet other Fellows as well as top Facebook researchers.

Third, and it seems not many people know this, the program is open to PhD students from all around the world, with no limit per university.

Q: What kind of research is Facebook interested in supporting?

DW: Research at Facebook is typically grounded in real-world problems. Research teams work on cutting-edge topics with a practical focus, which ultimately means that focus areas could include multiple disciplines. Consider that the Facebook family includes Instagram and WhatsApp (as well as others), which could result in various products within human-computer interaction, computer vision, privacy, or data science. For a more detailed list, I would recommend looking at the list of available fellowships on the Fellowship page.

Q: How do you write a research statement?

DW: Start by taking some time to really think about the topic you are proposing. This would involve reading up on the latest publications but also borrowing from an inspiration to solve real-world problems around you. In your first draft, do not focus on the word limit. Rather, try to effectively communicate the problem and why it matters. Afterward, you could work on reframing and then editing to adhere to the word limit. Generally, I recommend following the structure below:

Paragraph 1: Introduction

  • Present the problem
  • Identify who this impacts and why this is relevant in general and more specifically relevant to the company
  • One sentence summarizing your idea/approach

Paragraph 2: Body

  • What you plan to do
  • How you plan to do it
  • What you’ve done to show you can do this (optional)

Paragraph 3: Conclusion

  • Contribution to the community (academic and public)
  • Relevance to the mission/values of the company

Q: What advice would you provide with regard to the application process?

DW: Having ample time always works in your favor. Therefore, starting to plan earlier rather than later would be in your best interest. However, don’t let this discourage you if you find out about an opportunity close to its deadline. My high-level advice would be the following:

  • Ensure that your research statement is on a topic you are passionate about and that you clearly communicate that passion. When I applied for fellowships in 2018, I had two complete sets of applications prepared for submission. Both were well-motivated and important work. In the end, my adviser recommended that I choose the one that I was without a doubt most passionate about. Ultimately, that application was successful. Being able to communicate your passion could help to convince others why your research direction is worthy of a fellowship award.
  • Apply to multiple fellowships. I could insert multiple cliches to stress that “it’s a numbers game” and that “you shouldn’t place all your eggs in one basket.” Fellowships are very competitive. I applied twice before being awarded the Facebook Fellowship, and I received Google’s Women TechMakers Scholarship on the third try. I recommend creating a document or spreadsheet with possible options to help you manage.
  • Feel free to reach out to past fellows. I’ve had numerous students reach out to me for advice, and I try to provide as much help as I can. You could also look at the type of research that is normally conducted by past fellows to get a sense of what that organization might be interested in. However, keep in mind that some companies like Facebook are rapidly evolving and interests might change year to year.

To learn more about Wilkinson’s background, research interests, publications, and speaking experiences, visit her Fellowship profile.

The post Fellowship 101: Facebook Fellow Daricia Wilkinson outlines the basics for PhDs appeared first on Facebook Research.

Read More

Facebook awards $200,000 to 2020 Internet Defense Prize winners at USENIX Security

by Facebook AI

This week, Facebook and USENIX awarded a total of $200,000 to the top three winners of the Internet Defense Prize at the 29th USENIX Security Symposium. Created in 2014, the award is funded by Facebook and offered in partnership with USENIX to celebrate contributions to the protection and defense of the internet.

This year we awarded a $100,000 first-place prize to Sathvik Prasad, Elijah Bouma-Sims, Athishay Kiran Mylappan, and Bradley Reaves at North Carolina State University for their work titled “Who’s Calling? Characterizing Robocalls through Audio and Metadata Analysis.” The paper discusses an 11-month “honeypot” study the team conducted to understand robocalls. The team’s findings surface potential technological and policy solutions to combat robocalling.

The second-place prize of $60,000 was awarded to Adam Oest (Arizona State University), Penghui Zhang (Arizona State University), Brad Wardman (PayPal, Inc.), Eric Nunes (PayPal, Inc.), Jakub Burgis (PayPal, Inc.), Ali Zand (Google), Kurt Thomas (Google), Adam Doupé (Arizona State University), and Gail-Joon Ahn (Samsung Research) for their paper, “Sunrise to Sunset: Analyzing the End-to-End Life Cycle and Effectiveness of Phishing Attacks at Scale.” This paper explores the impact of real-world phishing attacks on customers of a financial institution.

Our third-place prize of $40,000 went to Emily Tseng (Cornell University), Rosanna Bellini (Open Lab, Newcastle University), Nora McDonald (University of Maryland, Baltimore County), Matan Danos (Weizmann Institute of Science), Rachel Greenstadt (New York University), Damon McCoy (New York University), Nicola Dell (Cornell Tech), and Thomas Ristenpart (Cornell Tech) for their research titled, “The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums.” This paper takes a deeper look into how victims of intimate partner violence are surveilled.

We’d like to congratulate the 2020 winners of the Internet Defense Prize and thank them for their contributions to help make the internet more secure. To learn more about the Internet Defense Prize and past winners, please visit the Internet Defense Prize website.

The post Facebook awards $200,000 to 2020 Internet Defense Prize winners at USENIX Security appeared first on Facebook Research.

Read More

Recommendations for researchers to more accurately measure time spent on Facebook

Recommendations for researchers to more accurately measure time spent on Facebook

by Facebook AI

People often ask whether spending time on social media is good or bad for us. To answer this question, researchers need accurate ways to measure how much time people spend on platforms like Facebook, among other things. The most common approach, found in the vast majority of published studies, is through survey questions asking participants how much time they spent on these platforms. However, participants’ reports of their own use have well-documented limitations. Participants may not report accurately, because they either can’t recall or don’t know. Keeping track of time is hard, and people may report in biased or skewed ways. Some people may be more prone to recall errors. Further, validating these self-report measures is challenging in the absence of data from internal server logs.

Our aim is to provide researchers with validated self-report time measures that more closely capture people’s actual time spent on Facebook. In our latest paper, “How Well Do People Report Time Spent on Facebook? An Evaluation of Established Survey Questions with Recommendations” (CHI 2020), we evaluate common survey questions from the literature, provide recommendations to researchers, and provide translations for 14 languages.

We compared data from 10 self-reported Facebook-use survey measures deployed in 15 countries (N = 49,934) against data from Facebook’s server logs. We found that:

  • Participants significantly overestimated how much time they spent on Facebook and underestimated the number of times they visited. For example, on one survey question, people overestimated how much time they spent on Facebook by an average of 3.2 hours per day (see Figure 1).
  • Self-reported time spent was only moderately correlated with actual Facebook use (r = 0.23–0.42 across the 10 questions).
  • Some questions caused underestimation, while others caused overestimation. Only 27 percent were able to respond accurately even on the best-performing question.
  • The more time people spent on Facebook, the more likely it was that they misreported their time.
  • Teens and young adults have more error reporting their time on Facebook, which is notable because of the high reliance on college-aged samples in many fields.

Figure 1

Participants asked “How many hours a day, if any, do you typically spend using Facebook?” overestimated their time on Facebook by an average of 3.2 hours per day.

Informed by these results, we recommend the following to researchers aiming to measure time spent on Facebook:

1. To reduce measurement error, we recommend that researchers ask participants to report data from time management tools like Your Time on Facebook rather than try to estimate it themselves.

2. When time spent must be collected via self-report, we recommend the following wording from Ellison et al. (2007), which had the lowest error in our study.

  • In the past week, on average, approximately how much time PER DAY have you spent actively using Facebook?
    • Less than 10 minutes per day
    • 10–30 minutes per day
    • 31–60 minutes per day
    • 1–2 hours per day
    • 2–3 hours per day
    • More than 3 hours per day

3. Because self-reports of time spent are imprecise, we suggest that researchers not use these values directly but rather interpret people’s self-reported time spent as a noisy estimate of where they fall on a distribution relative to other respondents.

While our focus here is “time spent” questions, because these are very common in the literature, a growing body of studies shows that merely examining the amount of time an individual uses social media is inadequate for many questions of interest (such as how social media use might be associated with loneliness, social comparison, or academic performance). Instead, we recommend focusing on how people use social media, as discussed in the following studies:

  1. The relationship between Facebook use and well-being depends on communication type and tie strength by Moira Burke and Robert E. Kraut
  2. Social capital and resource requests on Facebook by Nicole B. Ellison, Rebecca Gray, Cliff Lampe, and Andrew T. Fiore
  3. Do social network sites enhance or undermine subjective well-being? A critical review. by Philippe Verduyn, Oscar Ybarra, Maxime Résibois, John Jonides, and Ethan Kross

Beyond these implications to researchers, we hope tools such as Your Time on Facebook provide people with more insight into the time they spend on our platform, and foster conversations around their perceptions of use and online habits. Connecting our own perceptions of Facebook use (“How is the time I spend on Facebook good/bad for me?”) and what scientific research tells us about social media’s impact on our lives is also crucial. In that regard, we hope insights from our study provide readers with tools to critically engage with science communication on social media use and well-being.

With the evolution of the platform with new features and shifts in how people use Facebook, even the strongest survey measures are likely to evolve. That said, employing a stable set of established measures is an important methodological practice for researchers to support comparative work within the scientific community. We hope to make a positive contribution by providing such validated measurements that support international, academic, and comparative work on the impact of Facebook on people’s lives.

The post Recommendations for researchers to more accurately measure time spent on Facebook appeared first on Facebook Research.

Read More

Population mobility, small business closures, and layoffs during the COVID-19 pandemic

Population mobility, small business closures, and layoffs during the COVID-19 pandemic

by Facebook AI

Global findings from the Future of Business Survey and Facebook Movement Range Maps

Since the onset of the COVID-19 pandemic, Facebook’s Data for Good Program has been sharing insights with nonprofits, researchers, and public health officials to support the global response. Data for Good shares aggregate statistics on things like whether people are generally staying put in response to stay-at-home policies, as well as perspectives shared from our online community of 150 million businesses about how the pandemic has affected their operations. Using data from several Facebook data sets, we examine the extent to which population mobility influences business outcomes. We find that declines in country-level mobility are heavily correlated with a higher share of small and medium businesses (SMBs) on Facebook reporting layoffs, as well as with the proportion of small businesses having completely closed due to the pandemic.

Data sources

Future of Business Survey

The Future of Business Survey is an ongoing collaboration between Facebook Data for Good, the Organisation for Economic Co-operation and Development (OECD), and the World Bank to survey online small and medium businesses on the Facebook platform about their conditions, challenges, and operations. To provide timely information in response to the COVID-19 outbreak, the Future of Business has shifted to a monthly sampling frame that aims to assess SMBs’ responses to the pandemic in more than 50 countries.

In May, the Future of Business surveyed over 30,000 small businesses around the world and found that, during the pandemic, more than one in four had closed and one in three had laid off workers. In June, we conducted a follow-up survey among 25,000 small business owners and managers and found that as many countries had begun to ease their lockdown restrictions, some businesses were able to resume their in-person operations but nearly one in five (18 percent) businesses remained closed.

Movement Range Maps

Part of Facebook’s Disease Prevention Maps toolkit, Movement Range Maps are intended to inform researchers and public health experts about how populations are responding to physical distancing measures. To analyze how population mobility shifts as stay-at-home orders are put into place, these maps calculate a “change in movement” metric, which looks at how much people are moving around and compares it with a baseline period that predates social distancing measures. This data is derived from people who are using Facebook on a mobile device and who have opted in to the Location History feature. When publishing Movement Range Maps, we aggregate observations to a county level and add random noise to protect privacy.

Analysis

Effects on employment

To compute a weighted average of relative change of mobility for each country, we took the publicly available movement range data by region and divided it by the number of observations in each subnational unit in the country. We then analyzed the correlation between relative changes in mobility and small business layoffs at the country level as reported in the Future of Business for the month of June, examining businesses that reported having been fully closed as well as businesses that remained open. We see that the percent of businesses that laid off employees is correlated with drops in mobility (coefficient = –0.54) and that a higher proportion of small businesses in sub-Saharan Africa and Latin America laid off workers as compared with those in the European region.

To check for robustness, we also fit a simple linear regression, including the region of the country as a fixed effect to see whether the relationship between mobility rates and layoffs remained after controlling for geographic influences. When we control for region, the estimated coefficient of relative mobility remains negative (–0.42) and statistically significant (p < 0.01), suggesting that country-level declines in mobility have a unique and significant effect on small business layoffs even when controlling for a broader set of regional factors.

We then analyzed the correlation between relative changes in mobility during the month of June and small businesses closures. This analysis revealed that countries with the lowest levels of mobility had more businesses closed during the pandemic and countries with higher overall mobility had the fewer closures (coefficient = –0.73).

When we include regional fixed effects, the estimated coefficient of the mobility change was –0.41 and statistically significant (p < 0.001), suggesting that every percentage point drop in mobility in June was associated with a 0.41 percentage point increase in the business closure rates during the pandemic, independent of regional influences.

Conclusion

Analyzing June data from the Future of Business Survey and Movement Range Maps, we find that declines in mobility are strongly correlated with layoffs as well as business closure rates at a country level. These findings suggest that as states, cities, and countries face COVID-19 outbreaks and corresponding lockdowns, small businesses will continue to experience closures and layoffs. As a result, the small business community is likely to continue to need support over the coming year from local and international institutions that are seeking to help business owners mitigate the effects of the pandemic.

Data from this research blog, including the Future of Business Survey and Movement Range Maps, is shared publicly in an effort to better help respond to the COVID-19 pandemic. To access Facebook’s publicly available data sets, please visit our page on Humanitarian Data Exchange.

The post Population mobility, small business closures, and layoffs during the COVID-19 pandemic appeared first on Facebook Research.

Read More

Facebook launches new research award opportunity focused on digital privacy

by Facebook AI

Facebook is deeply committed to honoring people’s privacy in our products, policies, and services. Part of that commitment involves empowering the academic community to pursue research in this area in order to broaden our collective understanding of global privacy expectations and experiences. With this commitment in mind, Facebook launched the first of a series of privacy-related research award opportunities in November 2019, followed by a second opportunity in January 2020. As a continuation of this series, Facebook now invites the academic community to respond to the People’s Expectations and Experiences with Digital Privacy request for proposals (RFP).
View RFP“This research award opportunity seeks applications from across the social sciences and technical disciplines, and encourages collaboration between fields,” said Liz Keneski, Head of Privacy Research at Facebook. “We are keen to engage with academia in this cross-disciplinary space to inform the creation of world-class privacy experiences for our users.”

Disciplines include but are not limited to anthropology, communications, computer science, economics, engineering, human-computer interaction, human factors, political science, social psychology, and sociology. We are particularly interested in proposals that focus on the following two areas:

  1. Advancing understanding of users’ privacy attitudes, concerns, preferences, needs, behaviors, and outcomes
  2. Informing novel interventions for digital transparency and control that are meaningful for diverse populations, contexts, and data types

Applications for this RFP are now open. Deadline to apply is Wednesday, September 16, at 5:00 p.m. AOE. For more information about areas of interest, proposal requirements, eligibility, and more, visit the application page.

The post Facebook launches new research award opportunity focused on digital privacy appeared first on Facebook Research.

Read More

Announcing the winners of Facebook’s request for proposals on misinformation and polarization

by Facebook AI

Misinformation and polarization are fundamental challenges we face, not just as a company with the mission of bringing people together but also as members of societies dealing with layered challenges ranging from election interference to a global pandemic.

At the end of February, Facebook Research launched a request for proposals focusing on these dual challenges. Our goal is to support independent research that will contribute to the understanding of these phenomena and, in the long term, help us improve our policies, interventions, and tooling. We invited proposals that took any of a wide variety of research approaches to bring new perspectives into ongoing work on issues like health misinformation, affective polarization, digital literacy, and more.

We received over 1,000 proposals from 600 institutions and 77 countries around the world that covered an impressive range of disciplines and methodological approaches. The 25 awardees intend to investigate these issues across 42 countries: Argentina, Australia, Brazil, Canada, Chile, China, Colombia, Denmark, Egypt, Ethiopia, Germany, Ghana, Hungary, India, Indonesia, Israel, Italy, Kenya, Mexico, Myanmar, New Zealand, Nigeria, Pakistan, Philippines, Russia, Rwanda, Spain, South Africa, South Korea, Sudan, Taiwan, Tanzania, Thailand, Tunisia, Turkey, Uganda, Ukraine, United Arab Emirates, United Kingdom, United States, Vietnam, and Zimbabwe.

Proposals were evaluated by a selection committee comprising members of Facebook’s research and policy teams. The selection process was incredibly competitive, so we want to thank all the researchers who took the time to submit a proposal. Congratulations to the winners.

Research award winners

The names listed below are the principal investigators of each proposal.

Affective polarization and contentious politics: Women’s movement in Mexico
Marta Barbara Ochman, Instituto Tecnológico y de Estudios Superiores de Monterrey

Affective polarization: Causal drivers, online networks, and Interventions
Selim Erdem Aytaç, Koç University

Can third party fact-checkers on Facebook reduce affective polarization?
Fei Shen, City University of Hong Kong

Countering deepfake misinformation among low digital-literacy populations
Ayesha Ali, Lahore University of Management Sciences

Cross-cultural psychological motivations of online political hostility
Michael Bang Petersen, Aarhus University

Dangerous speech, social media and violence in Ethiopia
Mercy Fekadu Mulugeta, Addis Ababa University

Digital literacy and misinformation among smallholder farmers in Tanzania
Justin Kalisti Urassa, Sokoine University of Agriculture

Digital literacy, demographics and misinformation in Myanmar
Hilary Oliva Faxon, Phandeeyar

Digital literacy in East Africa: A three country comparative study
Meghan Sobel Cohen, Regis University

Digital literacy in Latin America: Developing measures for WhatsApp
Kevin Munger, Pennsylvania State University

Do online video recommendation algorithms increase affective polarization?
Brandon Stewart, Princeton University

Do users in India, Kenya, Ghana react differently to problematic content?
Godfred Bokpin, CUTS Accra

Examining how ingroup dissent on social media mitigates false polarization
Victoria A. Parker, Wilfrid Laurier University

Exploring harmful [mis]information via normalized online violent content
Joanne Lloyd, University of Wolverhampton

Indigenous women and LBGTQI+ people and violence on Facebook
Bronwyn Carlson, Macquarie University

Micro-Influencers as digital community health workers
Kathryn Cottingham, Dartmouth College

Political elites and the appeal of fake news in Brazil
Natália Salgado Bueno, Emory University

Political identity ownership
Shannon C. McGregor, University of North Carolina at Chapel Hill

Quantifying harms of misinformation during the U.S. presidential election
Erik C. Nisbet, Northwestern University

Quantifying persistent effects of misinformation via neural signals
Joseph W. Kable, University of Pennsylvania

STOP! Selective trust originates polarization
Sergio Splendore, Universitá degli Studi di Milano

The circulation of dangerous speech in the 2020 Brazilian elections
Lucas Calil Guimarães Silva, Fundação Getúlio Vargas

The contagion of misinformation
Heidi Larson, London School of Hygiene & Tropical Medicine

Unpacking trust and bias in social media news in developing countries
Denis Stukal, University of Sydney

When online speech meets offline harm: Internet shutdowns in Africa
Nicole Stremlau, University of Oxford

—

To view our currently open research awards and to subscribe to our email list, visit our Research Awards page.

The post Announcing the winners of Facebook’s request for proposals on misinformation and polarization appeared first on Facebook Research.

Read More

The six most common Fellowship questions, answered by Facebook Fellow Moses Namara

by Facebook AI

Since 2013, the Facebook Fellowship Program has supported bright and talented PhD students from around the world who are engaged in innovative research. This year, on August 10, we will once again invite PhD students to apply for the upcoming 2021 Fellowship cohort. In preparation for this next round of applications, we connected with Moses Namara, 2020 Fellow in Privacy and Data Use, to learn more about his experience applying to become a Fellow.

Namara is a PhD candidate in human-centered computing at Clemson University, advised by Dr. Bart Knijnenburg on privacy decision-making research. Namara first became involved in the Facebook Fellowship Program in 2017, when he won the Emerging Scholar Award. Before returning to Clemson for the 2020–2021 academic year, he completed a summer internship at Facebook on the UX Research team.

In this Q&A, Namara offers advice about writing a research statement, navigating the application process, being a Facebook Fellow, and knowing whether you’re qualified to apply.

Q: How did you decide on a research topic for your research statement?

Moses Namara: My decision was driven by my research interests, prior work, and what research questions I wanted to address. This process involved doing a literature review of the research topic to learn what others had done and identify existing gaps that I could address. However, this was done in relation to one or more of the available Fellowships listed on the Facebook Fellowship page.

To ensure that my topic was applicable to Facebook, I read blog posts and articles relevant to areas where my research topic could apply. Based on what I learned from this process, I came up with a plan, which I shared with my academic adviser and peers to ensure that it was both applicable to Facebook and academically feasible to do. After this feedback, I started drafting my research statement. In a nutshell, I identified a research topic based on a combination of my research interests, skill set, the importance of the topic to my research field, and its applicability to Facebook.

Q: What are some questions that I should try to answer while writing my research statement?

MN: For a concrete research statement, there are four key questions that you should try to answer in one or two sentences before you write out a full-fledged statement:

  1. What are you trying to do?
  2. How is it done today, and what are the limits of the current practice?
  3. What’s new in your approach, and why do you think it will be successful?
  4. Who cares about it? If you are successful, what difference will it make?

Thinking hard about these questions will help sharpen your ideas and hopefully help you produce a quality research statement.

Q: What advice would you provide with regard to the application process?

MN: Start to work on your research statement early enough so that you can receive feedback and continue to iterate on it up to a point where you are confident and happy with it.

Ensure that your final research statement is well written, grammatically correct, concise, and easy to read and comprehend, especially for people who may not be as familiar with your area of research.

Make sure that you get recommendations from someone familiar with your work, and ask for them well in advance. This will give the recommenders ample time to write high-quality recommendations for you, which they can’t do at the last minute.

Feel free to reach out to past Fellows; they are always happy to share their experiences and provide tips on how to go about the process. Ask them if they are able to review and provide feedback on your statement if they have the time. This can be an opportunity to hear from someone who has gone through the process, and also to get better insight about the Fellowship.

To help make your research applicable to Facebook, read some of the research that they have published within your area, read their research blog to identify new products or research challenges they are trying to address, and read the Fellowship FAQs in case you have any questions.

Q: What are the other benefits of this Fellowship, apart from the stipend and tuition award?

MN: The greatest benefit is the network that you would be able to form with other Fellows — especially within your cohort. These students come from some of the top universities in the world and are people who could potentially end up as your friends or collaborators.

Another benefit is being invited to the annual Facebook Fellowship Summit, where you get to meet and interact with some of the smartest people in technology who work at Facebook. This is an advantage because graduate school not only involves conducting high-quality research but also requires the ability to network and champion your work if it is to be known and/or impactful. The Summit is virtual this year, but it’s still a good opportunity to connect.

The Fellowship also allows us the freedom to work on any research project we choose, as opposed to one that needs to be funded.

Lastly, the conference funds help you to attend conferences you might be interested in whether you have a paper at those conferences or not, thus offering you an opportunity to network, and meet and interact with other people within or outside of your research area.

These are all great benefits just beyond the financial help. It is also important to note that the internship/employment process is separate from the Fellowship.

Q: What are some of the things you can expect from being a Facebook Fellow?

MN: Beyond the monetary compensation, you can expect to meet highly intelligent, passionate students just like yourself at the annual Fellowship Summit (whether it’s virtual or not). At the Summit, you are likely to meet recruiters and people who work in industry on applying the same concepts within your research area.

The Facebook Fellowship is a big opportunity for you as a graduate student. As with every opportunity, it is up to you to make the best of it. There is nothing extra expected of you, and your research agenda is driven entirely by you, without any external influence or expectation from Facebook. Actually, that freedom puts the onus on you — to make the best of this opportunity to further your career or next step, in and out of graduate school.

Q: How can I tell if I’m qualified?

MN: Anyone eligible to apply is qualified. It doesn’t matter if you go to a lesser-known university: As long as it’s accredited, you qualify. It doesn’t matter if you are from a developing nation: As long as you are attending university, you qualify. It doesn’t matter if you’re doing research in something like engineering or psychology: As long as it’s related to one or more of the Fellowships that are available, you qualify.

I encourage you to apply regardless of the background you come from, whether you are enrolled in a university located in Africa, Asia, the Middle East, Europe, Latin America, or North America. Even if you are at the South Pole — if you are an eligible PhD student, you are qualified and should apply! It is very easy to do so since you do not have to go through your university. If at first you don’t succeed, then try again the next year. Your application will keep improving each time.

As a graduate student, you’re poised to become a researcher or an independent scientist who will have to work to get your research ideas funded. My advice is to take the opportunity now, because participating in the application process means you gain experience writing a research plan/proposal — something that not all graduate students explicitly get to do during the course of their studies.

The most important tip I have is that you do submit your application — and on time!

—

To learn more about Moses Namara’s background, research interests, publications, and speaking experiences, visit his webpage.

The post The six most common Fellowship questions, answered by Facebook Fellow Moses Namara appeared first on Facebook Research.

Read More