Senior Machine Learning Engineer

About the role

You will own the design and implementation of next-generation AI agents and agentic assist systems that directly power real-time augmentation for customer-facing agents. You will architect and deliver intelligent, multi-step agent workflows that unify knowledge retrieval, reasoning, and automated actions into cohesive production pipelines. You will build and evaluate LLM-powered systems, taking responsibility for defining quality metrics such as accuracy, faithfulness, and task completion across complex, non-deterministic use cases. You will diagnose and mitigate critical failure modes including hallucinations, retrieval errors, tool misuse, context drift, and prompt brittleness while ensuring reliability at scale. You will lead the development of insights platforms grounded in enterprise data through retrieval-augmented generation and large language model pipelines. You will collaborate closely with product, frontend, and backend teams to integrate AI capabilities seamlessly into Cresta's platform and customer workflows. You will mentor engineers, contribute to technical strategy, and help shape the roadmap for Cresta's AI-driven customer experiences. You will translate cutting-edge research into scalable, production-grade systems that drive measurable business impact across every channel.

Key facts

What you'll do

• Lead the design and development of Cresta's next-generation AI Agents and Agentic Assist systems, defining system architecture and core modeling approaches.
• Architect intelligent, multi-step agent workflows that combine real-time guidance, knowledge retrieval, reasoning, summarization, and automated actions into cohesive production systems.
• Design, deploy, and optimize LLM-powered systems, including Retrieval-Augmented Generation (RAG) pipelines, multi-agent orchestration, and domain-adapted models.
• Improve reasoning, planning, and tool-use capabilities in real-world AI applications across distributed environments and customer contexts.
• Develop evaluation strategies for complex, non-deterministic systems, including offline benchmarking, online experimentation, and LLM-as-a-judge methodologies.
• Diagnose and mitigate real-world failure modes such as hallucinations, retrieval errors, tool misuse, prompt brittleness, and multi-step reasoning breakdowns.
• Define and measure quality metrics (e.g., accuracy, faithfulness, task completion, latency, cost, robustness) to improve system reliability and performance.
• Optimize AI systems for scalability, latency, security, and cost efficiency in production environments.
• Collaborate cross-functionally with product, frontend, and backend teams to integrate AI capabilities seamlessly into Cresta's platform.
• Mentor engineers, contribute to technical strategy, and help shape the roadmap for Cresta's AI systems.
• Conduct experiments, analyze results, and iterate rapidly to validate hypotheses and drive measurable business outcomes.
• Partner with stakeholders to translate customer needs and business objectives into technical requirements for AI solutions.
• Implement monitoring and observability for deployed models to ensure consistent behavior and rapid issue detection.
• Explore and prototype emerging techniques to maintain Cresta's leadership position in agentic AI and conversational intelligence.

Requirements

• Hold a Bachelor's, Master's, or PhD degree in Computer Science, Machine Learning, or a related technical field, or have equivalent practical experience.
• Demonstrate substantial experience with modern machine learning frameworks and production-level model deployment.
• Show deep expertise in Large Language Models, including architecture, training techniques, and inference optimization.
• Possess a strong background in building and scaling retrieval-augmented generation and information retrieval systems.
• Have a proven track record of designing agentic systems and multi-step reasoning workflows in real applications.
• Exhibit strong problem-solving skills for diagnosing and fixing complex AI system failures in production.
• Bring experience with defining and tracking quantitative metrics for AI quality, accuracy, and performance.
• Have excellent communication skills to collaborate effectively with cross-functional teams and articulate technical concepts to diverse stakeholders.

Nice to have

• Experience with conversational AI platforms and customer contact center environments.
• Knowledge of industry standards and best practices for evaluating non-deterministic AI systems.
• Familiarity with modern MLOps tooling, monitoring, and deployment pipelines for AI at scale.

Practical notes

This is a full-time position based in the United States and eligible for remote work. Only candidates who meet the stated requirements will be considered.