Member of Technical Staff

About the role

At Reflectionai, we are dedicated to advancing the field of artificial intelligence by developing open and accessible intelligence models. As a Member of Technical Staff, you will play a crucial role in constructing the essential data systems and infrastructure that support our research initiatives, training processes, and production environments. This position offers an exciting opportunity to contribute to groundbreaking projects in a collaborative and innovative setting.

Key facts

What you'll do

• Design and oversee the ingestion and orchestration frameworks necessary for both streaming and batch data processing.
• Build robust compute and storage infrastructures that include runtimes and engines capable of handling large-scale data processing tasks.
• Establish reproducible data pipelines that incorporate isolated execution, version control, and backfill functionalities to ensure data integrity.
• Implement comprehensive data governance practices, including lineage tracking and quality assurance standards, to guarantee reliable outcomes in production.
• Continuously monitor and optimize system performance and costs, employing strategies such as performance tuning and budget management.
• Integrate the data layer to facilitate a seamless model development lifecycle, bridging the gap between research and production stages.
• Collaborate with cross-functional teams to ensure alignment between data engineering efforts and research objectives.
• Stay updated on industry trends and emerging technologies to enhance our data systems and practices.

Requirements

• Demonstrated experience in data engineering, with a successful history of deploying production-grade data pipelines at scale.
• Proven expertise in designing and implementing comprehensive data systems that handle large volumes of batch and streaming data.
• Strong problem-solving skills, particularly in diagnosing and resolving complex pipeline issues while optimizing for both performance and cost.
• A proactive approach with a strong inclination towards building new systems from the ground up, showcasing a high-agency mindset.
• Excellent communication abilities, enabling effective collaboration with both research and infrastructure teams.
• A solid understanding of data governance principles and best practices in data management.

Nice to have

• A genuine passion for constructing the data backbone that supports open-weight AI systems.
• Familiarity with machine learning concepts and their application in data engineering.
• Experience with cloud platforms and services related to data storage and processing.

Skills & tools

• Compute & Orchestration: Proficient in tools such as Spark, Flink, Beam, Airflow, Dagster, Kafka, and PubSub for data processing and orchestration.
• Storage & Analytics: Knowledgeable in data lake and warehouse architectures, including technologies like Parquet, Iceberg, Delta Lake, BigQuery, and Snowflake.
• Metadata & Data Quality: Experienced in managing lineage systems, metadata management, and tools like Great Expectations to ensure data quality and reproducibility.
• Cost & Performance Management: Skilled in implementing partitioning strategies, clustering techniques, and cost optimization methods to enhance the efficiency of data pipelines.

Practical notes

• The compensation package includes a competitive salary along with equity options.
• We offer a comprehensive benefits package that includes medical, dental, vision, and life insurance, as well as an annual wellness allowance.
• Employees enjoy daily lunch and dinner provided at the office to foster a collaborative environment.
• Our parental leave policy offers 22 weeks of paid leave for all parents, supporting work-life balance.
• We provide unlimited paid time off for employees based in the U.S., promoting a healthy work-life integration.
• Visa sponsorship is available for qualified candidates, ensuring that we can attract top talent from diverse backgrounds.
• Please note that this position may be subject to U.S. Export Administration Regulations, which govern access to specific technologies or source code.

Join us at Reflectionai, where your expertise will help shape the future of artificial intelligence through innovative data solutions. If you are ready to take on this challenge and contribute to meaningful advancements in the field, we encourage you to apply.