Lead Machine Learning Inference Engineer, Advertising

About the role

You will architect and manage a high-performance inference platform designed to meet the rigorous latency and throughput demands of our advertising ecosystem. This leadership position involves guiding technical strategy across hardware, software, and model optimization to support our global streaming platform.

Key facts

What you'll do

• Architect and build a high-performance inference platform.
• Create observability and monitoring tools to maintain system reliability and model performance.
• Diagnose and eliminate system bottlenecks to ensure efficient end-to-end operation.
• Evaluate and integrate new developments in ML hardware acceleration and inference frameworks.

Requirements

• M.S. degree or higher in CS, ECE, or a related technical field.
• 10+ years of experience building large-scale distributed systems.
• 5+ years of experience in a technical lead or leadership capacity.
• Proficiency in high-performance programming languages.
• Expertise in ML system deployment and inference frameworks.
• Demonstrated success in optimizing large-scale ML systems, including GPU acceleration, HPC techniques, and hardware-software co-design.
• Experience leading cross-functional, global teams focused on low-latency, high-throughput ML serving.

Nice to have

• Documented contributions to open-source systems or machine learning projects.

Skills & tools

• Machine Learning Inference
• Distributed Systems
• High-Performance Computing
• GPU Acceleration
• Hardware-Software Co-design

Practical notes

• Salary range: $246,500 - $486,100 annually for California residents.
• Hybrid work policy: In-office attendance is generally expected Monday through Thursday, with flexible remote work on Fridays.
• Benefits include equity awards, health insurance (medical, dental, vision), 401(k), disability, life insurance, parental leave, and wellness resources.
• Reasonable accommodations for the hiring process can be requested via EmployeeRelations@Roku.com.