Senior Product Manager, Experimentation Tooling
Job description
About the role
You will lead the end-to-end product strategy for experimentation and post-training workflows, guiding the roadmap from initial concept through to market launch and adoption. This role focuses on streamlining how AI researchers and engineers validate models, manage fine-tuning, and move work from research into production environments.
Key facts
What you'll do
- Define the product vision and roadmap for post-training and experimentation features.
- Analyze current ML engineering workflows to identify friction points in debugging, model comparison, and handoffs.
- Partner with engineering to shape technical solutions, architecture, and implementation.
- Establish model evaluation as a core product function, using quantitative and qualitative signals to guide development.
- Collaborate with Finance and Growth teams to design pricing and packaging strategies.
- Drive go-to-market efforts, including technical sales support, positioning, and developer-focused content.
- Instrument and track key metrics such as iteration speed, compute usage, retention, and expansion.
- Use the platform directly to identify and resolve workflow bottlenecks.
Requirements
- 7+ years of product management experience, including at least 3 years in developer tools, infrastructure, platform, or machine learning products.
- Hands-on experience building for AI researchers, data scientists, or ML engineers.
- Deep knowledge of post-training and experimentation, including distributed training, hyperparameter optimization, artifact lineage, and model evaluation.
- Proficiency in modern post-training techniques like reinforcement learning, supervised fine-tuning, or preference optimization.
- Technical depth to discuss APIs, SDKs, observability, and distributed systems with engineering teams.
- Proven ability to simplify complex technical workflows into intuitive user experiences.
- Experience managing pricing, unit economics, or consumption-based product models.
- BS in Computer Science, Engineering, or equivalent practical experience.
Nice to have
- Experience at an AI infrastructure, neocloud, hyperscaler, or experiment-tracking company.
- Familiarity with PyTorch, PyTorch Lightning, GPU infrastructure, or large-scale fine-tuning.
- Experience with tools such as Hugging Face, Weights & Biases, MLflow, Ray, Slurm, or Kubernetes.
Skills & tools
- Product strategy and roadmap execution
- ML experimentation and evaluation frameworks
- Developer-facing API and SDK design
- Pricing and GTM strategy
- Distributed training and model lifecycle management
Practical notes
Total rewards include a discretionary bonus, equity (RSUs), and comprehensive benefits. Benefits include medical, dental, and vision coverage, 401(k) matching, unlimited PTO, paid parental leave, an annual professional development allowance, and a two-week company-wide winter break.
About the company
Lightning AI is a technology company that develops tools and platforms for artificial intelligence development. The company takes its name from the English Electric Lightning, a British fighter aircraft that served as an interceptor from 1960 to 1988. The aircraft was capable of a top speed above Mach