Staff Engineer - Experimentation Team

About the role

Join LaunchDarkly's Experimentation team to develop the core platform enabling data-driven product decisions. You will engineer systems that allow customers to conduct A/B tests, measure feature impact, and optimize user experiences, all integrated with a feature management system handling trillions of daily evaluations. This role blends data science and platform engineering, focusing on statistical engines, data warehouse analysis pipelines, and advanced experimentation methods.

Key facts

What you'll do

Develop the statistical engine for experimentation, including hypothesis testing, sequential analysis, variance reduction techniques like CUPED, and power analysis, ensuring statistical accuracy for all experiment types.

Design and build data warehouse-native experimentation capabilities that execute analysis directly within customer data warehouses such as Snowflake, Databricks, Redshift, and BigQuery, creating adaptable, warehouse-agnostic computation layers.

Lead the development of adaptive experimentation systems, incorporating machine learning models like contextual bandits and Bayesian optimization for allocation strategies beyond simple A/B tests.

Collaborate with product management, design, and data science to shape the product and technical direction for the experimentation platform.

Partner with other engineering teams, including Warehouse Integrations, SDK, Platform, and Data Science, to ensure cohesive product development.

Mentor fellow engineers, elevating the team's standards for statistical rigor and system architecture.

Take ownership of operational aspects, including monitoring, observability, incident response, and on-call duties, ensuring system reliability.

Requirements

A minimum of 10 years of experience building large-scale experimentation platforms, statistical analysis systems, or data-intensive backend services.

Demonstrated knowledge of applied statistics, covering hypothesis testing, sequential analysis, variance reduction (e.g., CUPED), power analysis, and experiment design, with an understanding of frequentist versus Bayesian approaches.

Experience with machine learning for adaptive experimentation, such as contextual bandits, Thompson sampling, Bayesian optimization, or reinforcement learning-based allocation.

Proven ability to design systems that function across different data warehouses like Snowflake, Databricks, Redshift, and BigQuery.

Proficiency in backend development languages such as Go or Python, suitable for service development and statistical computation.

Experience with event-driven architectures, data pipelines, and processing large volumes of data.

Familiarity with cloud environments (AWS, GCP) and infrastructure-as-code practices.

A history of technical leadership, including setting technical direction, problem decomposition, and influencing across teams.

The ability to clearly communicate complex statistical concepts to both product and engineering stakeholders.

Nice to have

Experience with modern data warehousing concepts and distributed computing frameworks.

Skills & tools

Go, Python, Snowflake, Databricks, Redshift, BigQuery, AWS, GCP, Infrastructure as Code, Event-driven architectures, Data Pipelines, Machine Learning, Applied Statistics.

Practical notes

Compensation:

Zone 1 (San Francisco/Bay Area, NYC Metro, Boston, Seattle): $214,800 - $295,350

Zone 2 (Irvine, LA, Monterey, Santa Barbara, Santa Rosa, Austin, Portland, Philadelphia, Chicago): $193,400 - $265,870

Zone 3 (All other US locations): $182,600 - $251,020

Exact compensation is determined by skills, experience, and location.

Benefits include Restricted Stock Units (RSUs), health, vision, and dental insurance, and mental health benefits.

LaunchDarkly is an equal opportunity employer.

Recruiters will only contact candidates from @launchdarkly.com email addresses or verified LinkedIn accounts. Be wary of unsolicited requests for money or banking information.