Data Engineering Lead

About the role

THG is seeking an experienced Data Engineering Lead to join its team in Manchester, UK. This role is critical for advancing the company's data engineering capabilities, which underpin daily analytics, business intelligence, and AI-driven personalization efforts. The successful candidate will be responsible for leading the design, development, and evolution of data pipelines and architecture, ensuring that data across various domains such as Customer, Commercial, and Operations is reliable, accessible, and scalable. This position involves a combination of technical leadership, hands-on engineering, and strategic planning, working closely with cross-functional teams to deliver high-quality data solutions that support the company's growth and innovation objectives.

Key facts

What you'll do

• Lead the architecture and design of data pipelines and systems, establishing best practices for data ingestion, transformation, and storage across multiple domains including Customer, Commercial, and Operations.
• Develop, implement, and maintain standards for data management, ensuring data quality, consistency, and integrity across all datasets.
• Drive the modernization of the data platform, focusing on the BigQuery and Google Cloud Platform (GCP), to support scalable, efficient, and reliable data pipelines.
• Make strategic architectural decisions, balancing technical feasibility with business needs, and ensuring the solutions are robust and future-proof.
• Build structured, trustworthy datasets within each domain, working towards a unified source of truth that enables accurate data reconciliation, reporting, and analysis.
• Promote the integration of AI tools and techniques into data engineering workflows to enhance pipeline automation, data quality monitoring, and anomaly detection.
• Collaborate closely with the Ingenuity engineering team to ensure seamless data integration, effective data sharing, and high-quality data outputs.
• Lead, mentor, and develop a team of six data engineers, fostering a culture of technical excellence, continuous learning, and innovation.
• Establish clear goals and performance metrics for the data engineering team, providing guidance and support to ensure delivery of high-quality solutions.
• Stay current with emerging trends and technologies in data engineering, evaluating their applicability to the company's needs and integrating them as appropriate.
• Engage with stakeholders across the business to understand their data requirements, translating these into scalable data solutions that support analytics and decision-making.
• Ensure compliance with data governance policies and collaborate with governance teams to uphold data security, privacy, and regulatory standards.
• Document data architecture, pipelines, and processes thoroughly, ensuring maintainability and knowledge sharing within the team.
• Participate in planning and prioritization of data engineering projects, balancing technical debt with new feature development.

Requirements

• Extensive hands-on experience working with BigQuery and GCP, including designing, building, and maintaining production data pipelines.
• Strong proficiency in Python, with experience in scripting, automation, and data transformation tasks.
• Proven experience working within at least two of the following data domains: Customer, Commercial, and Operations, with a deep understanding of their data needs and challenges.
• Demonstrated ability to evolve data architecture and improve data management practices over time, with a focus on scalability and reliability.
• Solid understanding of data modeling principles, data pipeline design, and distributed data systems.
• Familiarity with data transformation tools such as Dataform or dbt, and their application in managing complex data workflows.
• Experience applying AI tools and techniques within data engineering contexts, such as automated quality checks, anomaly detection, or pipeline optimization.
• Proven leadership skills, including mentoring, team management, and fostering a collaborative environment.
• Strong problem-solving skills and the ability to translate complex business requirements into technical solutions.
• Excellent communication skills, capable of working effectively with technical and non-technical stakeholders.
• Knowledge of data governance principles and experience working with governance teams to ensure compliance and data security.
• Ability to document technical processes clearly and thoroughly for future reference and team knowledge sharing.

Nice to have

• Experience in building Customer Data Platforms or similar data consolidation projects, integrating multiple data sources into a unified platform.
• Familiarity with real-time data processing and streaming pipelines, enabling real-time analytics and decision-making.
• Industry experience in e-commerce or retail sectors, understanding the specific data challenges and opportunities within these industries.
• Knowledge of data privacy regulations and best practices for data security and compliance.
• Experience working in agile development environments, participating in sprint planning and iterative delivery.

Skills & tools

• BigQuery, GCP, Python, Dataform, dbt, data modeling, data architecture, AI tools for data engineering, data governance.

Practical notes

THG is committed to fostering a diverse and inclusive workplace. We welcome applications from candidates of all backgrounds and experiences. If you require any adjustments or accommodations during the recruitment process, please inform us. We aim to respond to all applications within 14 days. The company values continuous learning and innovation, encouraging team members to stay current with emerging technologies and industry best practices. This role offers an exciting opportunity to lead impactful data initiatives in a dynamic, fast-growing organization, supporting strategic decision-making and operational excellence across the business.