Senior Data Engineer
Job description
Senior Data Engineer at 3pillar Global
About the role
In this position, you will play a crucial role in developing and maintaining data pipelines that support our clients' needs. You will work with a variety of data technologies and collaborate with a global team to drive data-driven solutions. This role emphasizes both technical expertise and the ability to mentor junior team members.
Key facts
What you'll do
- Analyze business requirements to design effective data solutions.
- Write and implement code for proposed data strategies.
- Create and manage data pipelines, including version control and change management.
- Navigate the complexities of versioned data pipelines.
- Develop ETL/ELT processes for data extraction and manipulation from various sources.
- Design, build, and maintain both batch and real-time data pipelines in a production environment.
- Automate workflows related to data ingestion, aggregation, and ETL processing.
- Implement logging and monitoring for pipelines and services.
- Ensure data accuracy, integrity, privacy, security, and compliance through quality control measures.
- Provide training and mentorship to junior engineers.
Requirements
- At least 5 years of experience in a data engineering role or similar capacity.
- Strong proficiency in SQL and experience with relational databases and their design.
- Familiarity with cloud data warehouse solutions such as Snowflake, Redshift, BigQuery, or Azure Synapse.
- Proficient in Python, with hands-on experience using libraries like Pandas and NumPy.
- Experience with big data technologies such as Spark, MapReduce, Hadoop, and Hive.
- Skilled in data pipeline and workflow management tools like Airflow.
- Experience with data visualization tools such as Power BI, Tableau, or AWS QuickSight.
- Knowledge of data engineering services offered by AWS, Azure, or GCP.
- Understanding of AWS services including S3, Lambda, EMR, Glue ETL, Athena, RDS, Redshift, EC2, and IAM, or Azure services like ADF, Azure Synapse Analytics, ADLS Gen2, and Azure SQL DB.
- Solid experience with Data Lakes and Data Warehouses.
- Excellent problem-solving, communication, and organizational abilities.
- Proven capability to work independently as well as collaboratively within a team.
- Ability to mentor and guide fellow data engineers.
Nice to have
- Experience with data ingestion tools such as Fivetran, Stitch, or Matillion.
- Understanding of NoSQL databases like Redis, Cassandra, MongoDB, or Neo4j.
- Familiarity with machine learning technologies.
- Experience with Java or Scala.
Skills & tools
- SQL, Python, Airflow, Spark, AWS, Azure, GCP, Power BI, Tableau, Snowflake, Redshift, BigQuery, Hadoop, Hive, Fivetran, NoSQL databases.
Practical notes
This position is open to candidates who require visa sponsorship. The company offers a remote-first work environment, promoting flexibility and a focus on employee well-being. Benefits include opportunities for professional development and a supportive global community. Applications are encouraged from individuals who meet the qualifications listed above.
About the company
Ranorex is a software development company that provides Ranorex Studio, a GUI test automation framework. The framework supports the testing of desktop applications, web-based applications, and mobile applications. The company focuses on delivering tools that help development teams verify software quality across multiple platforms. Its technology enables users to create and execute automated tests without requiring extensive programming knowledge. The platform includes features for object recognition, test recording, and cross-browser execution. Ranorex serves organizations that need to accelerate release cycles while maintaining high reliability standards. The software integrates with common continuous integration servers and issue tracking systems to fit into existing development workflows. Customers use the framework to reduce manual testing effort and increase test coverage for complex user interfaces. The company continues to develop its automation capabilities to address evolving application architectures and device ecosystems. Ranorex operates in the software testing market alongside other vendors that specialize in quality assurance solutions. Its tools are designed for professional QA engineers and developers who build enterprise-grade applications. The business model centers on licensing the automation framework and providing technical support to license holders.