Sr. Staff Platform/Data Reliability Engineer, Databricks

About the role

Shield AI is a venture-backed defense-technology company dedicated to protecting service members and civilians through intelligent systems. The organization develops and operates a portfolio of advanced technologies. These include the Hivemind autonomy software platform, the V-BAT and X-BAT aircraft systems, and the Aechelon suite of simulation and synthetic reality tools. With operational facilities across the United States, Europe, the Middle East, and the Asia-Pacific region, Shield AI supports missions globally. More information is available at www.shield.ai, and the company maintains a presence on LinkedIn, X, Instagram, and YouTube. This Sr. Staff Platform/Data Reliability Engineer role centered on Databricks is responsible for the integrity and performance of the data platforms that underpin these defense systems. You will own the design and reliability of the data intake and transformation pipelines that feed critical flight and simulation telemetry. Success in this position requires collaboration with software engineers who build the Hivemind autonomy capabilities and the Aechelon simulation tools. Your work will directly support the operational readiness and testing cadence of the V-BAT and X-BAT aircraft systems.

Key facts

What you'll do

• Architect and maintain data ingestion pipelines that capture high-volume telemetry and sensor information from distributed test assets.
• Develop transformation layers that standardize heterogeneous flight log data into consistent formats for consumption by analysis and engineering teams.
• Conduct in-depth reviews of schema modifications and usage patterns to identify and mitigate future operational risks before they impact production.
• Implement robust monitoring and alerting frameworks that provide clear, real-time visibility into platform health for defense operations operators.
• Coordinate closely with teams responsible for simulation and synthetic reality environments to ensure temporal and structural data alignment across datasets.
• Establish and enforce validation tests that confirm data quality, completeness, and fidelity across geographically distributed collection and test points.
• Author and maintain comprehensive documentation for the data interfaces and APIs used by Hivemind and Aechelon development groups.
• Partner on experimental initiatives that analyze the behavioral characteristics and performance data of V-BAT and X-BAT systems during flight regimes.
• Optimize data storage and processing workflows on the Databricks platform to handle escalating volumes of defense-related telemetry and log data.
• Implement data governance practices that ensure compliance with internal standards and the sensitivity associated with defense-related information.
• Troubleshoot complex data pipeline failures and performance bottlenecks with minimal disruption to critical testing and evaluation schedules.
• Collaborate with data scientists and engineers to ensure that the data platform supports advanced analytics and machine learning workloads.
• Proactively identify data anomalies and inconsistencies, driving root cause analysis and remediation strategies with upstream teams.
• Contribute to the technical roadmap for the data reliability and platform engineering functions in support of evolving product needs.

Requirements

• Must bring a minimum of three years of professional experience in platform engineering or data reliability roles within technically complex environments.
• Must possess advanced skill in writing and optimizing SQL queries against large and complex operational datasets that span multiple schemas.
• Must demonstrate a deep understanding of how distributed systems transport, serialize, and secure information between microservices in production environments.
• Must exhibit the ability to adhere to strict timelines and operate reliably when handling sensitive defense-related information and controlled unclassified information (CUI).
• Must have hands-on experience with data pipeline orchestration, monitoring, and alerting to ensure high availability and rapid incident response.
• Must be comfortable working with log-structured data and telemetry streams from physical systems such as aircraft and autonomous vehicles.
• Must show a strong commitment to documentation and knowledge sharing to ensure continuity and clarity for cross-functional teams.
• Must be able to obtain and maintain any required security clearances or comply with defense industry vetting processes as applicable to the role.

Nice to have

There are no specific Nice to Have qualifications listed for this role.

Practical notes

Candidates are reminded to verify details on the official application page.

What you'll do

• Meet the bar Staff Platform/Data Reliability Engineer, Databricks.

About the company

Shield AI, Inc. is an American aerospace and defense technology company based in San Diego, California. The company develops autonomous systems for military and commercial applications, focusing on AI-powered drones and aircraft.

Staff Platform/Data Reliability Engineer, Databricks at Shieldai.

Staff Platform/Data Reliability Engineer, Databricks at Shieldai.

Staff Platform/Data Reliability Engineer, Databricks at Shieldai.