Sr. Data Engineer I

About the role

As a , you will be a key contributor to the development and enhancement of our Scibids product. Your primary responsibilities will include building and maintaining data pipelines, integrating external partner APIs, and working closely with cross-functional teams to ensure the platform's data infrastructure supports our advertising solutions effectively. You will be involved in designing scalable backend services, troubleshooting production systems, and collaborating with various teams such as Data Science, Analytics, and Product to optimize our offerings. This role offers an exciting opportunity to work on large-scale data systems within a dynamic environment focused on digital advertising and programmatic auctions.

Key facts

What you'll do

• Investigate and integrate external partner APIs to facilitate the growth of the Scibids product, ensuring seamless data flow and compatibility.
• Collaborate with Product teams to understand the technical capabilities and limitations of partner platforms, helping to identify constraints early in the development process.
• Design, develop, and maintain backend services and endpoints that are used internally by other teams within DoubleVerify, ensuring they are reliable, scalable, and efficient.
• Build, optimize, and manage data pipelines and services that process, transform, and reconcile data from multiple sources, including partner platforms and internal systems.
• Handle large datasets generated from digital advertising auctions, ensuring data integrity, system reliability, and performance at scale.
• Work closely with Data Science, Analytics, Product, and Engineering teams to translate business requirements into technical solutions that support platform optimization and product growth.
• Maintain production systems by troubleshooting issues, diagnosing root causes, and implementing long-term solutions to prevent recurrence.
• Utilize automated deployment tools and infrastructure-as-code frameworks to enable continuous integration and continuous delivery (CI/CD) for new features and updates.
• Improve system observability, reliability, and maintainability by implementing monitoring, logging, and alerting best practices.
• Conduct code reviews, participate in system design discussions, and mentor junior engineers to foster a collaborative and high-quality engineering environment.
• Collaborate with AI platform teams to support deployment and integration of intelligent solutions, enhancing the platform's capabilities.
• Ensure compliance with data privacy and security standards, particularly when handling sensitive or regulated data.
• Document technical designs, APIs, and data workflows clearly for internal teams and stakeholders.
• Stay current with industry trends, emerging technologies, and best practices in data engineering and API integration.
• Contribute to the continuous improvement of engineering processes, tools, and methodologies within the team.

Requirements

• Minimum of 3 years of experience in data engineering, backend engineering, or related fields, demonstrating a strong foundation in building scalable data systems.
• Proficiency in Python and SQL, with experience in developing production-level data pipelines and backend services.
• Familiarity with integrating external APIs, especially in a high-scale environment, and developing services that are robust and maintainable.
• Ability to navigate ambiguous external platform behaviors and create effective internal abstractions to handle variability.
• Experience designing and managing data pipelines or services that process large datasets efficiently and reliably.
• Knowledge of cloud environments such as AWS, GCP, or Azure, and familiarity with modern engineering practices including Git, CI/CD pipelines, Docker, and container orchestration tools.
• Strong problem-solving skills and the ability to troubleshoot complex issues in production systems.
• Excellent communication skills, with the ability to work effectively with product managers, data scientists, and other technical teams.
• Demonstrated commitment to automation, system reliability, observability, and data-driven decision-making.
• Ability to work in a fast-paced environment, prioritize tasks effectively, and adapt to changing requirements.
• Experience with AI or machine learning deployment platforms is a plus, but not mandatory.
• Prior experience in AdTech, programmatic advertising, or auction-based systems is beneficial but not required.
• Understanding of data privacy standards and security best practices when handling sensitive data.

Nice to have

• Experience working with AI or machine learning deployment platforms to support intelligent features.
• Background in digital advertising, programmatic auctions, or related fields.
• Familiarity with large-scale data processing frameworks such as Apache Spark or Kafka.
• Knowledge of additional programming languages like Java or Rust.
• Experience working in a collaborative Agile environment with cross-functional teams.

Skills & tools

• Python
• SQL
• API integration
• Cloud platforms (AWS, GCP, Azure)
• Git version control
• CI/CD pipelines and automation tools
• Docker and container orchestration (Kubernetes or similar)
• Monitoring and observability tools (e.g., Prometheus, Grafana)
• Data pipeline frameworks (e.g., Apache Airflow)

Practical notes

The salary for this position will be determined based on the candidate's qualifications, skills, experience, and location. The estimated salary range is between $89,000 and $178,000. This role is also eligible for bonuses, equity, and benefits. We value diversity and encourage applicants from various backgrounds to apply, even if they do not meet every listed requirement. The position requires on-site presence at our New York, NY office, with a hybrid work schedule of three days per week on-site. We are committed to providing a supportive environment that fosters professional growth and innovation.