Senior Data Engineer - Azure Databricks

About the role

Bamboohr is on the lookout for a highly skilled Senior Data Engineer specializing in Azure Databricks to join our dynamic team. This role is crucial for driving our data engineering initiatives, leveraging the power of Azure Databricks to enhance our data processing capabilities. As a Senior Data Engineer, you will be responsible for designing and implementing robust data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver data-driven solutions. Your expertise will play a pivotal role in transforming our data landscape and supporting our business objectives.

Key facts

What you'll do

• Design, develop, and maintain scalable data pipelines using Azure Databricks to facilitate efficient data processing and analytics.
• Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets that meet their needs.
• Implement data quality checks and monitoring processes to ensure the integrity and accuracy of data throughout the pipeline.
• Optimize existing data workflows for performance and cost efficiency, leveraging the capabilities of Azure Databricks.
• Work closely with stakeholders to gather requirements and translate them into technical specifications for data solutions.
• Participate in code reviews and provide mentorship to junior data engineers to promote best practices in data engineering.
• Document data engineering processes, architectures, and workflows to ensure knowledge sharing and compliance with best practices.
• Stay updated on the latest trends and technologies in data engineering and Azure Databricks to continuously improve our data capabilities.
• Troubleshoot and resolve data-related issues in a timely manner, ensuring minimal disruption to business operations.
• Collaborate with DevOps teams to integrate data solutions into CI/CD pipelines, enhancing deployment efficiency.
• Engage in performance tuning and optimization of data storage solutions within Azure to maximize efficiency and reduce costs.
• Contribute to the development of data governance frameworks to ensure compliance with data privacy regulations and organizational policies.

Requirements

• Proven experience as a Data Engineer, with a focus on Azure Databricks and related technologies.
• Strong proficiency in SQL and experience with data modeling and database design principles.
• Familiarity with programming languages such as Python or Scala for data processing and manipulation.
• Experience with data warehousing concepts and technologies, including ETL processes and data integration techniques.
• Knowledge of cloud computing concepts, particularly within the Azure ecosystem, including Azure Data Lake and Azure SQL Database.
• Understanding of data governance, data quality, and data security best practices.
• Excellent problem-solving skills and the ability to work independently as well as collaboratively in a team environment.
• Strong communication skills, capable of conveying complex technical concepts to non-technical stakeholders.
• A degree in Computer Science, Engineering, or a related field is preferred.

Nice to have

• Experience with other Azure services such as Azure Synapse Analytics or Azure Machine Learning.
• Familiarity with big data technologies like Apache Spark or Hadoop.
• Knowledge of data visualization tools such as Power BI or Tableau.
• Experience with Agile methodologies and working in a fast-paced environment.
• Certifications in Azure Data Engineering or related fields would be advantageous.

Skills & tools

• Azure Databricks
• SQL
• Python or Scala
• Azure Data Lake
• Azure SQL Database
• ETL tools
• Data modeling and warehousing
• Apache Spark
• Data governance frameworks

Practical notes

• This position is offered as a contract role, with the specific duration and terms to be discussed during the interview process.
• The salary for this role is competitive and commensurate with experience, details of which will be provided to shortlisted candidates.
• The exact location for this role is currently unspecified, and candidates should be prepared for potential remote work or travel requirements.
• Visa sponsorship may be available for qualified candidates, subject to the company's policies and regulations.

META

Company: Bamboohr

Title: Senior Data Engineer - Azure Databricks

Listed

location: Unknown

Job type: Contract

Apply URL: https://billigence.bamboohr.com/careers/574

Tags: Azure, Engineering