Principal AI Research Scientist, Research Director

About the role

This leadership position focuses on advancing large-scale machine learning, specifically targeting LLM training, inference efficiency, and model optimization. You will guide a team of researchers and engineers to develop novel scaling techniques that translate into production-ready capabilities for the Databricks AI platform.

Key facts

What you'll do

• Direct a multidisciplinary team in foundational and applied research regarding LLM scaling, systems performance, and efficiency.
• Create and execute a research roadmap that aligns with company objectives for foundation model development.
• Develop algorithmic innovations in neural network training, including low-precision methods, novel optimizers, and model adaptation.
• Collaborate with infrastructure and systems teams to improve distributed training, memory management, and compute utilization.
• Oversee the design of large-scale experiments and benchmark findings against current industry standards.
• Partner with product and engineering departments to integrate research breakthroughs into the Databricks AI platform.
• Represent the company at academic conferences and within the open-source community.
• Mentor research scientists and engineers on technical execution and career progression.

Requirements

• Proven experience leading research teams to develop techniques for foundation model efficiency with a track record of industry impact.
• Deep technical knowledge in at least one of the following: LLMs, generative AI, model optimization, distributed ML systems, or responsible AI.
• Strong programming proficiency in Python and PyTorch for research implementation and prototyping.
• Demonstrated ability to move research innovations into scalable product features.
• Strong communication skills with the ability to influence cross-functional roadmaps and stakeholders.

Nice to have

• Experience at the intersection of ML and systems, such as compiler optimization, kernel optimization, or distributed training frameworks.
• A strong network within the large-scale ML community, including service as a program committee member or area chair at conferences like NeurIPS, ICML, ICLR, or MLSys.
• A track record of research impact, such as first-author publications at top-tier conferences or significant open-source contributions.

Skills & tools

• Python
• PyTorch
• Distributed ML systems
• Large Language Models (LLM)
• Model optimization
• Distributed training frameworks

Practical notes

Compensation includes base salary, eligibility for an annual performance bonus, and equity. Databricks provides comprehensive benefits and maintains a commitment to equal employment opportunity and inclusive hiring practices.

About the company

Databricks is an American software enterprise headquartered in San Francisco, California. The company focuses on building technology for data processing and artificial intelligence applications. The organization was founded in 2013 by the original computer science researchers who created Apache Spark at the University of California, Berkeley. The company offers a cloud-based platform for advanced data analytics and artificial intelligence workloads. This environment operates natively across major cloud infrastructure providers, including Amazon Web Services, Microsoft Azure, and Google Cloud Platform. Databricks pioneered the data lakehouse architecture, a design that integrates features from traditional data warehouses and flexible data lakes to process both structured and unstructured data in one place. For data storage and exchange, Databricks created Delta Lake, an open-source project designed to add reliable ACID transaction support to data lakes. The platform also hosts an open data marketplace built on the Delta Sharing protocol. Serving as a managed artificial intelligence infrastructure provider, Databricks supplies proprietary foundation models inside a protected security perimeter. Users can run models from OpenAI, Anthropic, and Google Gemini directly through the system. The platform incorporates a variety of core data and artificial intelligence products. Lakebase operates as a specialized database built for artificial intelligence agents. Lakeflow Designer provides a tool for building automated data pipelines. Agent Bricks serves as a workspace to construct production-scale artificial intelligence agents. Additionally, Databricks recently introduced Genie Code, an autonomous artificial intelligence agent that helps users with data engineering, data science, and analytics assignments.