Data Engineer II
PditechnologiesIndia4d ago
Engineeringremotecurated-jd
Job description
Data Engineer II at Pditechnologies.
About the role
Join our team to build and manage cloud-based data pipelines that power analytics and business intelligence for global retail and petroleum brands. You will focus on integrating enterprise systems into our central data architecture while ensuring high performance and data accuracy.
Key facts
What you'll do
- Architect and support cloud data pipelines for enterprise reporting.
- Create and refine ETL and ELT workflows using AWS services.
- Connect business applications like Salesforce and NetSuite to our core data platforms.
- Construct scalable data models to facilitate business intelligence.
- Track pipeline health, resolve data discrepancies, and boost system reliability.
- Translate business reporting needs into functional technical designs.
- Uphold data governance through documentation and metadata management.
- Advance automation, CI/CD, and infrastructure-as-code workflows.
- Review code and mentor team members on engineering standards.
Requirements
- Bachelor degree in Computer Science, Engineering, Information Systems, or equivalent work experience.
- 2-4 years of professional experience in cloud data platform development or data engineering.
- Proficiency in AWS services including S3, Glue, Redshift, AppFlow, and Step Functions.
- Strong coding skills in Python, PySpark, and SQL.
- Knowledge of data warehousing, dimensional modeling, and data lake architecture.
- Experience syncing data from SaaS platforms like NetSuite or Salesforce.
- Familiarity with Git, CI/CD, and Infrastructure as Code, specifically Terraform.
- Ability to manage complex projects, communicate with stakeholders, and maintain accountability.
Nice to have
- Practical application of AI and machine learning for data engineering tasks, such as anomaly detection or automated quality monitoring.
Skills & tools
- AWS (S3, Glue, Redshift, AppFlow, Step Functions)
- Python
- PySpark
- SQL
- Salesforce
- NetSuite
- Git
- CI/CD
- Terraform