Data Engineer IV

About the role

As a senior Data Engineer, you will design and build scalable, cloud-based data solutions for analytics and business intelligence. You will lead complex data integration projects, define engineering standards, and guide other engineers. This role requires a deep understanding of modern data engineering and architectural principles.

Key facts

What you'll do

• Design and build scalable cloud data pipelines for enterprise analytics and reporting.
• Lead the creation of complex data integration solutions using AWS technologies.
• Develop scalable data models, ingestion frameworks, and transformation processes for reliable analytics.
• Apply data architecture principles to design secure and maintainable data pipelines and integration solutions.
• Establish engineering standards, reusable frameworks, and best practices to improve platform reliability.
• Work with business stakeholders to convert reporting requirements into technical solutions.
• Drive continuous improvement through automation, infrastructure as code, CI/CD, and DevOps.
• Lead troubleshooting for complex data platform, integration, and performance issues.
• Mentor other data engineers through code reviews and knowledge sharing.
• Evaluate new technologies and recommend improvements for platform performance and efficiency.
• Support data governance by promoting data quality and metadata management across the platform.

Requirements

• Bachelor's degree in Computer Science, IT, Engineering, or a related field, or equivalent experience.
• 8+ years of experience in data engineering, cloud data platforms, or related software engineering roles.
• Expert experience developing cloud data solutions with AWS technologies like S3, Glue, Redshift, AppFlow, Lake Formation, and Step Functions.
• Advanced proficiency with SQL, Python, and PySpark for ETL/ELT solutions.
• Practical experience applying AI/ML techniques to data engineering problems, such as automated data quality checks or anomaly detection.
• Practical experience developing semantic and contextual layers within a data lakehouse.
• Strong understanding of dimensional modeling, data warehousing, data lakehouse concepts, Apache Iceberg, and Parquet.
• Experience integrating enterprise platforms such as Salesforce and NetSuite.
• Strong understanding of modern data engineering practices and conceptual knowledge of data architecture principles.
• Experience with Infrastructure as Code (Terraform), Git, Azure DevOps, and CI/CD pipelines.
• Demonstrated ability to mentor engineers, influence technical decisions, and drive engineering best practices.
• Strong analytical, problem-solving, and communication skills for cross-team collaboration.

Skills & tools

• AWS (S3, Glue, Redshift, AppFlow, Lake Formation, Step Functions)
• SQL
• Python
• PySpark
• Apache Iceberg
• Parquet
• Terraform
• Git
• Azure DevOps
• CI/CD

Practical notes

PDI offers a competitive salary, market-competitive benefits, and a quarterly perks program. Employees receive ample time off and access to continuous learning, professional certifications, and leadership development. Hybrid working arrangements are available where appropriate.