Senior Staff Data Scientist - Consumer Relevance

Responsibilities

• Serve as the technical authority on relevance metrics and evaluation methodology across Consumer, setting standards for how we measure the quality of feeds, search results, and recommendations in a complex, community-driven environment
• Develop metrics frameworks and offline evaluation approaches for ranking and recommendation systems, including proxy metrics that reliably predict long-term outcomes like retention, community health, and user satisfaction
• Design and analyze experiments for relevance features, accounting for challenges unique to networked platforms such as spillover effects between communities, interference between contributors and consumers, and long-run impacts of ranking changes on content supply
• Identify opportunities where improved measurement and analysis can unlock product insights that were previously unmeasurable or ambiguous, particularly around content quality, search intent understanding, and personalization effectiveness
• Partner deeply with ML engineers and product teams to translate model performance metrics into user-facing impact
• Influence the long-term product strategy for Feeds and Search by synthesizing insights from experimentation, observational analysis, and metric deep-dives into clear, actionable recommendations for senior leadership
• Mentor and elevate other data scientists across the organization on relevance evaluation, experimentation best practices for ranking systems, causal reasoning, and statistical rigor
• Publish and share methodological advances internally and, where appropriate, externally to contribute to the broader relevance, recommendation systems, and experimentation community

Required Qualifications

• Ph.D. in Statistics, Computer Science, Information Retrieval, Economics, or a related quantitative field with a strong focus on recommendation systems, ranking, causal inference, or evaluation methodology; or M.S. with equivalent depth of expertise
• For M.S. holders: 12+ years of industry experience in applied science, data science, or relevance/ranking-focused roles
• For Ph.D. holders: 8+ years of industry experience in applied science, data science, or relevance/ranking-focused roles
• Deep expertise in metrics design and evaluation for ranking and recommendation systems, including offline metrics and counterfactual evaluation
• Strong understanding of causal inference and experimentation methodology, including practical experience with challenges relevant to ranking systems such as novelty effects, position bias, long-run effect estimation, and ecosystem-level impacts
• Experience defining and validating quality metrics for content ranking, search, or recommendations at scale
• Strong theoretical grounding in experimental design, including power analysis, variance reduction techniques, and sequential testing as applied to relevance experiments
• Expert knowledge of SQL and proficiency in R and/or Python for statistical computing
• Demonstrated ability to influence product and organizational strategy through data-driven insights about content quality and user experience
• Excellent communication skills with the ability to explain nuanced statistical and ML concepts and tradeoffs to both technical and non-technical senior stakeholders
• Experience mentoring data scientists and building organizational capability in relevance evaluation and experimentation
• Comfortable in innovative and fast-paced environments with a bias toward action

Preferred Qualifications

• Published research or industry contributions in areas recommendation systems or causal inference for ranking
• Experience with social network or user-generated content platforms where community-level dynamics create non-trivial relevance and experimentation challenges