Solutions Engineer - Data Engineering

About the role

Join the Global Services Delivery Team to guide customers through the implementation of Apache Airflow. You will serve as a technical advisor for organizations ranging from early-stage startups to large enterprises, helping them build and scale data workflows. In this capacity, you will own the design and execution of technical engagements that validate architecture decisions and de-risk deployment strategies. You will act as the primary link between product expertise and client success, ensuring that every interaction moves the customer toward a production-ready state. The role demands a balance of deep technical craftsmanship and consultative communication to solve complex workflow orchestration challenges. You will leverage structured feedback from the field to influence the product roadmap and enhance platform usability. This position is ideal for someone who thrives on translating abstract requirements into concrete, scalable data engineering solutions.

Key facts

What you'll do

• Assume the role of a technical expert on Apache Airflow and the Astronomer platform for post-sales customers, diagnosing workflow issues and prescribing remediation paths.
• Architect scalable data engineering frameworks and provide guidance on Airflow best practices, focusing on modular design and operational resilience.
• Perform code refactoring assessments to improve customer data workflows and system configurations, highlighting inefficiencies and anti-patterns.
• Utilize AI-assisted development tools to speed up coding, testing, and troubleshooting while ensuring security and compliance standards are upheld.
• Gather structured feedback from client interactions to inform product improvements and feature development, translating user needs into actionable insights.
• Communicate technical solutions through email, Slack, and pair programming sessions, ensuring clarity and alignment across distributed teams.
• Conduct in-depth workflow audits that examine task flow, resource allocation, and dependency management to optimize performance.
• Collaborate with product managers and developers to reproduce complex issues and validate fixes within controlled test environments.
• Mentor junior engineers and data practitioners on Airflow fundamentals, fostering a culture of knowledge sharing within client organizations.
• Drive the delivery of proof-of-concept projects that demonstrate the value of managed orchestration and cloud-native integrations.
• Maintain a detailed understanding of the Astronomer platform features, ensuring that configuration options are leveraged to their full potential.
• Support the creation of technical artifacts such as runbooks, architecture diagrams, and deployment guides for ongoing reference.
• Evaluate the maturity of customer data pipelines and propose incremental improvements that align with industry standards.
• Act as a subject matter expert during discovery sessions, helping to scope work and define success criteria for engagements.

Requirements

• 3-5 years of experience in data engineering, specifically with Apache Airflow in production environments, demonstrating a history of maintaining critical workflows.
• Proficiency in writing Python and building custom Airflow operators and hooks to extend the functionality of the platform.
• Experience creating and managing DAGs, including version control, testing, and scheduling strategies that ensure reliability.
• Background in building, optimizing, and monitoring ETL/ELT pipelines, with an understanding of data quality and lineage.
• Familiarity with cloud platforms and data tools such as AWS, Azure, Google Cloud, Snowflake, Databricks, DBT, or Cloudera, and how they integrate with Airflow.
• Strong ability to prioritize tasks and manage customer-facing activities, balancing multiple engagements with competing deadlines.
• Excellent verbal and written communication skills, capable of explaining technical concepts to both technical and non-technical stakeholders.
• Demonstrated skill in troubleshooting complex pipeline failures and identifying root causes through log analysis and system inspection.
• Willingness to adhere to Astronomer's security policies and standard operational procedures during all client engagements.
• Commitment to continuous learning, keeping up with new releases of Airflow and related technologies in the data ecosystem.

Nice to have

• Experience with modern data stack ETL/ELT implementations in production, showing familiarity with transformation tools and orchestration patterns.
• Background in building and maintaining CI/CD pipelines, particularly those that automate testing and deployment of DAGs and configurations.
• Experience managing both short-term and long-term customer engagements, adapting communication style to suit different audiences.
• Proficiency with infrastructure technologies including Docker and Kubernetes, and understanding how they relate to managed Airflow deployments.

Practical notes

• This role follows a hybrid work model requiring at least 3 days per week at the Hyderabad office.
• Astronomer is an equal opportunity employer and does not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.