Research Engineer, Post-Training Inference

About the role

The Model Shaping team at Together AI builds products and research that enable customization of open foundation models for downstream applications. The team delivers services that help machine learning developers select appropriate models and improve them using domain-specific data. Members also design new methods for efficient model training and evaluation, drawing ideas from machine learning, natural language processing, and ML systems.

As a Research Engineer in Model Shaping, you will develop a platform that allows users to customize open-source models with their own data. You will work across training and inference stacks to build and improve Fine-Tuning, Reinforcement Learning, and Evaluation services. This includes creating a seamless path from post-training to production serving and optimizing the inference engine for reinforcement learning workloads. You will collaborate closely with product, research, and engineering teams to keep the API reliable, performant, and well integrated into the company's technical infrastructure. Your work will help build the foundational layer of the open-source AI ecosystem, enabling developers worldwide to efficiently create high-quality models tailored to their applications.

Key facts

Responsibilities

Design and build Together's systems for customizing open-source models.

Build integrations between the Model Shaping and Inference platforms to ensure a seamless path from post-training to serving production workloads.

Add features to inference engines for large-scale post-training experiments, including optimizations for reinforcement learning workloads.

Ensure service stability by participating in an on-call rotation and guaranteeing 24/7 availability of the platform.

Streamline evaluation workflows so experiments integrate smoothly into developer tooling.

Enable large-scale tuning by extending engine features for demanding training jobs.

Drive integration between training frameworks and serving infrastructure across teams.

Maintain API reliability through rigorous testing and continuous performance improvements.

Requirements

Have two or more years of experience building and deploying machine learning-based services in production environments.

Possess hands-on experience with modern inference engines such as SGLang, vLLM, and TensorRT-LLM.

Be familiar with the latest methods for fine-tuning LLMs and other AI models.

Have a strong software engineering background in Python or Go.

Stay up to date with the latest advances and trends in the machine learning community.

Experience that will make you stand out

Serve low-precision models such as FP4 or FP8, manage multiple LoRA adapters within one model instance, or handle models distributed across several GPU nodes.

Optimize the performance of reinforcement learning training workloads.

Develop CUDA, Triton, or CuTE DSL kernels for inference.

Develop large-scale and high-load production systems.

Maintain or contribute to open-source machine learning projects.

Manage machine learning workloads on Kubernetes clusters in production.

About Together AI

Together AI is a research-driven artificial intelligence company. The company believes open and transparent AI systems will drive innovation and create the best outcomes for society. Together AI is on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. The team has contributed to leading open-source research, models, and datasets to advance the frontier of AI. Its work has been behind technological advancements such as FlashAttention, ATLAS, RedPajama, and Mamba. The company invites you to join a passionate group of researchers in building the next generation AI infrastructure.

Compensation

The US base salary range for this full-time position is $200,000 - $290,000. Individual compensation will be determined by experience, skills, and job-related knowledge.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Practical notes

Please see the official apply page to confirm details.