Lead Compute Platform Engineer

About the role

As a , you will be an integral part of the Data Platform team, where your primary focus will be on the design, development, and maintenance of the data infrastructure that underpins the company's operations. This includes working on the lakehouse platform, which plays a crucial role in data processing. Your contributions will ensure that the data handling processes are both efficient and scalable, empowering engineers and analysts to tackle business challenges without the burden of infrastructure concerns.

Key facts

What you'll do

• Design and implement innovative features within the existing frameworks and services that the team oversees.
• Work closely with data scientists to improve their workflow and productivity through enhanced tools and systems.
• Identify opportunities for computational improvements and execute enhancements effectively.
• Engage in open-source contributions by providing patches for enhancements and bug fixes in technologies such as Spark, Iceberg, Hadoop, and Airflow.
• Act as a subject matter expert in Spark and Scala, sharing your knowledge with the team.
• Mentor junior engineers through collaborative coding sessions, design discussions, and knowledge-sharing initiatives to uplift the overall quality and adherence to best practices.
• Analyze system performance and propose optimizations to improve data processing efficiency.
• Collaborate with cross-functional teams to align data infrastructure with business objectives and requirements.
• Stay updated with industry trends and emerging technologies to ensure the team remains at the forefront of data engineering practices.
• Document processes, architectures, and workflows to facilitate knowledge transfer and onboarding of new team members.
• Participate in code reviews to maintain high standards of code quality and performance.
• Troubleshoot and resolve issues related to data processing and infrastructure in a timely manner.

Requirements

• A minimum of 7 years of experience in software engineering, demonstrating a strong background in developing scalable systems.
• A relevant degree in fields such as Computer Science, Software Engineering, Physics, Mathematics, or a similar area.
• At least 2 years of hands-on experience with Scala or other JVM-based programming languages, showcasing your technical proficiency.
• Familiarity with large-scale data processing frameworks, particularly Spark, is essential.
• Experience with container orchestration tools like Kubernetes is a plus.
• Basic knowledge of Amazon Web Services (AWS) components such as EC2, S3, and Athena is beneficial.
• A genuine interest in back-end technologies and architecture, coupled with a desire to continuously learn and grow.
• An affinity for mathematics, statistics, or adtech technology will be advantageous.
• A pragmatic and analytical mindset when approaching problem-solving tasks.
• Strong ability to work independently and adaptively in a fast-paced environment.
• Excellent communication and interpersonal skills, enabling you to articulate ideas and collaborate effectively with team members.

Nice to have

• Experience with data lakehouse architectures and their implementation.
• Familiarity with additional programming languages such as Python or Java.
• Knowledge of data governance and compliance best practices.
• Previous experience working in an agile development environment.
• Contributions to open-source projects or active participation in tech communities.

Skills & tools

• Proficient in Scala and Spark for large-scale data processing.
• Familiarity with cloud platforms, particularly AWS services.
• Experience with containerization and orchestration using Kubernetes.
• Understanding of data modeling and ETL processes.
• Knowledge of CI/CD practices and tools for continuous integration and deployment.
• Strong analytical skills and experience with performance tuning.

Practical notes

Zeta Global is dedicated to fostering a diverse and inclusive workplace. We encourage applicants from all walks of life and do not discriminate based on gender, race, color, religion, age, disability, veteran status, national or ethnic origin, sexual orientation, gender identity, or expression. Our aim is to cultivate a culture of trust and belonging, where everyone can bring their authentic selves to work and contribute to our collective success.

Salary range

The salary for this role is competitive, ranging from £100,000 to £112,000, depending on your experience and location.

About the company

 Zeta Global (NYSE: ZETA) is the AI-Powered Marketing Cloud that leverages advanced artificial intelligence (AI) and trillions of consumer signals to make it easier for marketers to acquire, grow, and retain customers more efficiently. Through the Zeta Marketing Platform (ZMP), our vision is to make sophisticated marketing simple by unifying identity, intelligence, and omnichannel activation into a single platform – powered by one of the industry’s largest proprietary databases and AI.