Member of Technical Staff - Machine Learning Capabilities

About the role

Preference Model is committed to pushing the boundaries of automated machine learning research engineering. The primary focus of this role is to develop and refine high-quality reinforcement learning environments that accurately reflect the complexities found in real-world scenarios. These environments are designed to enable frontier models to perform tasks that are typically handled by ML engineers or researchers, thereby accelerating the development and deployment of advanced AI systems. The position involves designing, building, and optimizing these environments, ensuring they provide meaningful signals for training and evaluation. The ideal candidate will have a strong background in ML research, a passion for creating innovative solutions, and the ability to work collaboratively with a multidisciplinary team to achieve ambitious goals.

Key facts

What you'll do

• Create and implement reinforcement learning environments tailored to produce clear, learnable reward signals that facilitate the training of frontier models in ML research and engineering tasks.
• Develop deep expertise in the latest advancements in ML research, including understanding and applying cutting-edge training and inference infrastructures.
• Collaborate closely with research and engineering teams to brainstorm, design, and develop new tools, techniques, and ideas that improve the environment-building process.
• Translate complex research insights into practical, scalable environment components that can be used to train large models effectively.
• Design and build complex, interactive RL environments that simulate real-world challenges, ensuring they are robust and scalable.
• Work with research teams to understand their specific needs and customize environments accordingly, enabling more efficient experimentation and evaluation.
• Optimize environment performance for scalability, efficiency, and reliability, ensuring they can handle large-scale training workloads.
• Stay current with the latest developments in ML, reinforcement learning, and related fields, incorporating new techniques and ideas into environment design.
• Contribute to documentation, best practices, and internal knowledge sharing to foster continuous improvement across teams.
• Participate in code reviews, testing, and debugging to maintain high-quality standards for environment development.
• Support deployment and integration of environments into existing ML workflows and infrastructure.
• Engage with the broader ML community through internal presentations and knowledge sharing to promote best practices and innovative ideas.

Requirements

• A minimum of 5 years of experience working in machine learning or research, with a focus on large language models (LLMs) and transformer architectures.
• Strong foundational knowledge in machine learning principles, with the ability to interpret and apply research papers, tutorials, and technical documentation to real-world RLVR challenges.
• Proficiency in Python programming, with experience in systems programming for performance-critical applications.
• Hands-on experience with either PyTorch or JAX frameworks for ML development.
• Demonstrated ability to proactively identify problems, develop solutions, and see projects through from conception to completion.
• Excellent problem-solving skills, with a focus on building scalable, efficient, and effective environments.
• Ability to work effectively in a fast-paced, evolving research environment, adapting to new challenges and priorities.
• Strong communication skills, capable of collaborating across teams and explaining complex concepts clearly.
• Commitment to continuous learning, staying updated on the latest ML research, infrastructure, and tooling developments.
• Experience working with large-scale ML infrastructure and environments is preferred but not mandatory.
• Ability to handle multiple projects simultaneously while maintaining high-quality standards.

Nice to have

• Advanced research experience with a track record of publications, open-source contributions, or significant projects in deep learning or ML fields.
• Formal research qualifications such as a PhD or MS degree in computer science, machine learning, or related fields are advantageous.
• Deep understanding of transformer architectures, including training and inference of modern large language models.
• Familiarity with inference libraries such as vLLM and SGLang, and experience integrating them into ML workflows.
• Strong skills in kernel development, including CUDA, Triton, or Pallas, to optimize performance-critical components.
• Experience designing and managing complex interactive RL environments, including simulation and real-world deployment scenarios.
• Knowledge of distributed training techniques and infrastructure for large-scale ML systems.

Skills & tools

• Python programming language, systems programming, PyTorch or JAX frameworks, reinforcement learning methodologies, transformer models, kernel development (CUDA, Triton, Pallas), research and development in ML.

Practical notes

• Visa sponsorship and relocation assistance are available to qualified candidates.
• The company offers comprehensive health, vision, and dental benefits, along with a 401K matching program to support employee well-being and financial planning.
• On-site daily lunch is provided, along with weekly snack deliveries to foster a collaborative and comfortable working environment.
• Preference Model values diversity and encourages applicants from all backgrounds to apply. If you find this role exciting but do not meet every single requirement, we still encourage you to submit an application.
• The role involves working on-site in San Francisco, with no remote or hybrid options unless explicitly stated.
• The position offers an opportunity to work at the forefront of ML research, contributing to impactful projects that shape the future of AI technology.