Data Engineering - AWS DMS
Job description
Data Engineering - AWS DMS at HCLTech.
About the role
HCLTech is in search of a skilled and driven Data Engineer specializing in AWS DMS. This full-time position focuses on enhancing technological advancements through innovation and creativity. We are only considering U.S. Citizens or Green Card Holders for this role.
Key facts
What you'll do
- Create services that facilitate data ingestion and synchronization with systems, ensuring near-real-time data access.
- Extract data from various sources utilizing Python and other ETL tools.
- Architect and implement an event-driven framework using AWS EventBridge, Kafka, or SNS/SQS for real-time data streaming.
- Design, build, and maintain scalable data pipelines that connect on-premises and AWS cloud environments.
- Write efficient Python scripts and applications using libraries like pandas and NumPy to process large datasets.
- Work with NoSQL databases such as MongoDB, Cassandra, and DynamoDB for high-performance data storage and retrieval.
- Develop and deploy applications in a cloud-native setup, utilizing modern cloud technologies for scalability and resilience.
- Monitor data workflows and systems continuously, troubleshoot issues, and optimize performance for reliability and scalability.
- Transition existing pipelines to MSSQL Server.
- Collaborate with business application owners on current data architecture, including data ingestion, pipelines, business logic, data consumption, and analytics needs.
- Design and document target data architecture, pipelines, processing, and analytics frameworks.
- Identify opportunities for optimization and consolidation.
- Work with the data team to break down business logic and data transformation patterns.
Requirements
- 7 to 15 years of experience in data engineering.
- Proficiency in AWS DMS, AWS Glue, Kafka, or SNS/SQS, Python/PySpark, data lakes, CloudWatch, CloudTrail, database design, and SQL.
- Familiarity with AWS IAM and EKS.
Nice to have
- Experience with data warehousing solutions.
- Knowledge of machine learning tools and frameworks.
Skills & tools
- AWS DMS
- AWS Glue
- Kafka
- SNS/SQS
- Python/PySpark
- Data lakes
- CloudWatch
- CloudTrail
- SQL
- NoSQL databases (MongoDB, Cassandra, DynamoDB)
Practical notes
- This position is open exclusively to U.S. Citizens or Green Card Holders.
- Salary range is between $78,000 and $148,000, depending on experience, skills, and education.
- This role may qualify for performance-based bonuses in accordance with company policies.
- Benefits include medical, dental, vision, pharmacy, life insurance, accidental death and dismemberment insurance, disability insurance, employee assistance programs, a 401(k) retirement plan, 10 days of paid time off annually, and 10 paid holidays per year.
How You'll Grow
At HCLTech, we provide ongoing opportunities for personal and professional growth. We aim to ensure you find fulfillment in your role while discovering what work truly inspires you. Our culture promotes open communication with senior staff, along with learning and career development programs at all levels. You will have the chance to explore different roles or pivot industries, empowering you to take charge of your career and find the position that suits you best.