Staff Statistics Engineer - Feature Flagging and Experimentation

The Opportunity

We are looking for a Stats Engineering leader to help us build two new Datadog products from scratch - Feature Flags and Experiments.

Our goal with these products is to help developers and product teams ship features quickly, experiment as second-nature, and make decisions with confidence. To do this, we need to build a world-class experimentation engine, backed by state-of-the-art statistical methods like sequential analysis, CUPED, and change point detection, which help to solve the big problems in the experimentation world of early peeking, long experiment durations, and catching bugs respectively.

Because there is often a technical, cultural, and linguistic gap between software engineers and statisticians, our Stats Engineers are unique in that they don't squarely fall into data science or software engineering. Data scientists typically understand the concepts, but may struggle to implement them in production with enterprise grade quality. Software engineers know how to build robust production systems, but can get lost implementing methods that don't have off the shelf frameworks. Here, we need the rare breed of builder who understands statistical concepts and can implement them in production.

Our experimentation platform will be used for root-cause analysis and decision-making across our 30,000+ customers of all shapes, sizes, and industries; we’ll help customers run everything from e-commerce-focused A/B tests on user adoption to infrastructure-based canary deployments in real-time to root cause major incidents, and connect the dots together across the worlds of the product manager, data person, and developer.

This is a rare opportunity to work with senior leaders across engineering, product, and design to define the foundational components of Datadog’s Product Analytics and APM stack from the ground up, and develop an experimentation engine in an AI-first world.

At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

What You’ll Do:

• Architect and implement the world-class experimentation engine behind Datadog Experiments, supporting methods like sequential testing, CUPED, variance reduction, and more
• Bring rigor to experiment analysis at scale (e.g. through diagnostics, guardrails for safe shipping)
• Translate complex statistical methods into robust, production-ready systems
• Work closely with Product, Design, and Engineering leadership to influence the direction of the product, on both a day-to-day and the big picture vision
• Educate engineers, leaders at Datadog, and our largest customers on statistical best practices, experiment design, and practical inference
• Help define standards and frameworks to make experimentation at D