Senior Data Platform Engineer

About the role

You will own the end to end design and operation of the observability and metadata layer for Chartis data platforms, ensuring that data assets are discoverable, trusted, and reliable. In this role, you will build and evolve the infrastructure that powers data lineage, monitoring, and search capabilities so that diverse teams can consume data with confidence. You will partner closely with platform, analytics, and security engineers to embed compliance and governance directly into the data lifecycle. You will design and maintain critical pipelines using tools such as Airflow or Dagster while ensuring high standards of quality and reliability. You will champion self service practices that empower internal and external users to find and understand data assets quickly. You will translate complex operational requirements into robust platform components that scale across healthcare use cases. You will mentor peers and junior engineers to elevate the entire organization's capability in data platform observability.

Key facts

What you'll do

Define and own the observability strategy for the data platform, including metrics, alerts, and dashboards that provide actionable insight into pipeline health and data quality.

Implement and operate orchestration frameworks such as Dagster or Airflow, designing robust DAGs and workflows that are observable, maintainable, and scalable in production environments.

Lead the implementation and governance of a data catalog solution, extending metadata coverage and improving discoverability of data assets through tools like DataHub.

Architect and manage data lifecycle controls, including retention, archival, and deletion policies, ensuring they align with regulatory requirements and client contractual obligations.

Ensure that data ingestion, storage, and processing practices meet healthcare compliance standards, with particular attention to PHI handling, audit logging, and encryption in transit and at rest.

Drive initiatives to improve platform performance, reliability, and cost efficiency, using observability signals to guide optimization efforts and capacity planning.

Build and maintain reusable Terraform modules and infrastructure as code pipelines that provision Azure resources for both firmwide analytics and client specific engagements.

Collaborate with product engineering, analytics delivery, and security teams to translate business and client requirements into technical roadmaps and execution plans.

Establish and enforce engineering standards for CI/CD, testing, documentation, and operational runbooks across data platform services.

Mentor junior engineers by providing guidance on platform best practices, debugging techniques, and observability driven development.

Champion a culture of data literacy and self service by creating knowledge sharing content and enabling cross functional training programs.

Partner with data scientists and analysts to ensure that data assets are well documented, versioned, and traceable through the full analytics lifecycle.

Continuously evaluate emerging tools and patterns in the data platform space, assessing their fit for adoption within Chartis' healthcare focused ecosystem.

Act as a technical leader in design reviews, incident response, and post incident reviews to improve long term platform resilience.

Requirements

You bring 7 or more years of professional experience in platform or data engineering roles, with a significant focus on cloud infrastructure and large scale data systems.

You have hands on experience with data catalog and metadata management platforms such as DataHub, Apache Atlas, or equivalent tools in production environments.

You demonstrate a strong understanding of data observability, metadata management, data lineage, and data governance principles and practices.

You have practical experience with cloud based data warehouses such as Snowflake or Databricks and with orchestration tools including Azure Data Factory, Dagster, or Airflow.

You are proficient in Python, using it to build maintainable scripts, pipeline components, and automation tools.

You have working knowledge of data ingestion patterns, including batch processing and change data capture concepts in real world scenarios.

You have experience establishing engineering standards, implementing CI/CD pipelines, and building observability into data platforms from the outset.

You have a proven track record of building and scaling platform components that enable self service analytics across multiple teams and use cases.

Nice to have

There are no preferred items listed in the source beyond the core qualifications and responsibilities.

Practical notes

This is a full time remote role. No specific hours, travel, visa, or application deadlines are stated in the source material.

LENGTH: 774 words