Data Engineer
onhiresBrazilFull Time1w ago
PythonAWSAzureGCPMongoDBClaudeAIETLAirflowSparkKafkaSQL
Job description
Data Engineer at onhires.
About the role
This position involves developing and enhancing a sophisticated data platform for the automotive industry. You will focus on creating robust data pipelines and improving data quality for millions of vehicle records. This is an opportunity to contribute to a modern data ecosystem within a remote-first technology company.
Key facts
What you'll do
- Construct and maintain data pipelines for both batch and streaming processes.
- Create dependable ETL/ELT workflows utilizing Python, Spark, and current orchestration tools.
- Enhance data quality, validation, monitoring, and observability across the platform.
- Design data models that support both analytical needs and product development.
- Build ingestion pipelines to connect various external and internal data sources.
- Improve processing speed, scalability, and overall platform reliability.
- Collaborate with product managers and software engineers to convert business needs into technical solutions.
- Utilize modern AI-assisted development tools to boost engineering efficiency.
Requirements
- At least 3 years of experience as a Data Engineer.
- Strong proficiency in Python and SQL.
- Practical experience with Spark/PySpark.
- Background in building production ETL/ELT pipelines.
- A solid grasp of data modeling concepts.
- Experience with workflow orchestration tools such as Airflow.
- Familiarity with cloud data platforms like AWS, Azure, or GCP.
- A strong sense of responsibility and the ability to work independently.
- Professional fluency in English.
Nice to have
- Experience with Databricks.
- Knowledge of Delta Lake or Lakehouse architecture.
- Familiarity with MongoDB or other NoSQL databases.
- Experience with CDC and streaming technologies such as Kafka or Debezium.
- Use of data quality frameworks like Great Expectations.
- Experience with metadata, lineage, or observability tools.
- Familiarity with AI-assisted development tools like GitHub Copilot, Cursor, or Claude Code.
Skills & tools
- Python
- SQL
- Spark/PySpark
- Airflow (or similar orchestration tools)
- AWS, Azure, GCP
- Databricks (nice to have)
- Delta Lake (nice to have)
- MongoDB (nice to have)
- Kafka, Debezium (nice to have)
- Great Expectations (nice to have)
- GitHub Copilot, Cursor, Claude Code (nice to have)
Practical notes
This is a remote position within Latin America. The company is based in Berlin and offers a long-term product with opportunities for professional growth.