Staff AI Engineer - AI Infrastructure & Agentic Platform

About the role

ShiftKey is evolving into an AI-centric healthcare marketplace, moving from initial trials to the establishment of a robust, production-ready infrastructure. This position focuses on taking charge of the architecture for our knowledge retrieval and agentic platforms, all while adhering to HIPAA regulations. You will spearhead the enhancement of our current retrieval systems and design the next generation of AI systems that utilize multi-step reasoning.

Key facts

What you'll do

• Enhance our AI knowledge platform by improving retrieval, indexing, and synthesis layers to better serve both internal engineering teams and customer-facing applications.
• Design and oversee the agentic infrastructure on AWS, ensuring that multi-step reasoning systems incorporate built-in cost management and observability features.
• Develop graph-based, relationship-aware retrieval systems that facilitate multi-hop queries and support the long-term accumulation of organizational knowledge.
• Define the API surface for the AI platform to create developer-friendly abstractions that assist product engineering teams in their efforts.
• Create reference implementations for agents that assist with operational incident triage and customer support, leveraging institutional data for accuracy.
• Oversee the cost model for AI infrastructure by tracking compute, model, and storage usage to ensure adherence to budgetary constraints.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications for AI systems.
• Conduct performance evaluations and optimizations of existing AI systems to ensure high efficiency and reliability.
• Mentor junior engineers and provide guidance on best practices in AI system design and implementation.
• Stay updated on industry trends and emerging technologies to continuously enhance the capabilities of our AI infrastructure.
• Participate in code reviews and contribute to the overall improvement of the engineering processes within the team.

Requirements

• A minimum of 7 years of professional experience in software engineering, with at least 2 years dedicated to the development and operation of production-grade AI or LLM systems.
• Strong expertise in retrieval engineering, particularly in hybrid search, query transformation, context-window management, and evaluation at a production level.
• Experience with agentic frameworks, including aspects such as tool utilization, iteration control, and trade-offs in model routing.
• Proficiency in programming languages such as Go, Python, or TypeScript, with an emphasis on strict typing, testing methodologies, and concurrency.
• Practical experience managing AI workloads on platforms like AWS Bedrock or Azure AI Foundry, including identity and secrets management.
• Capability to independently author and provision infrastructure using Terraform.
• Familiarity with observability tools tailored for AI, including metrics and structured logging focused on latency and expenditure.
• Knowledge of data handling protocols relevant to HIPAA or similar regulated SaaS environments.

Nice to have

• Previous experience in the healthcare sector, which would provide valuable context for the role.
• Familiarity with machine learning frameworks and libraries commonly used in AI development.
• Experience in working with large datasets and data engineering practices.

Skills & tools

• AWS (preferably Bedrock)
• Terraform
• Programming languages: Python, Go, or TypeScript
• Techniques such as Semantic RAG, HyDE, and re-ranking
• Understanding of agentic frameworks

Practical notes

• Employees are eligible for benefits, including health, vision, and dental coverage, starting from their first day of employment.
• The company offers a generous 401(k) plan with a 100% employer match up to 6%.
• Compensation packages include paid parental leave, generous paid time off, and company-recognized holidays.
• ShiftKey is committed to employee wellness, providing support and mental health resources.
• For any requests regarding reasonable accommodations due to disabilities, please reach out to accommodations@shiftkey.com.

This role represents a unique opportunity to be at the forefront of AI innovation within the healthcare industry, contributing to the development of systems that will shape the future of healthcare delivery. If you are passionate about leveraging AI to improve healthcare outcomes and meet the qualifications outlined, we encourage you to apply and join our dynamic team at ShiftKey.