Staff Quality Engineer

About the role

Staff Quality Engineer, Automation and Intelligence Pillar

StackAdapt is a technology company that enables marketers to reach, engage, and convert audiences with precision. The platform performs 465 billion automated optimizations per second, using AI to connect brand and performance marketing. Forward-thinking marketers use it to orchestrate high-impact campaigns across programmatic advertising and marketing channels. The company is currently seeking a Staff Quality Engineer to join the Automation and Intelligence Pillar pod. This pod is responsible for building StackAdapt's AI and agentic products. The role is a hands-on, individual-contributor leadership position. The successful candidate will set the technical direction for testing, evaluating, and shipping intelligent systems. The work spans LLM-powered features, agentic workflows, and the data and tooling that support them.

This position is remote-first. Candidates located anywhere in Canada and the United States are eligible to apply.

What You'll Be Doing

You will own and drive the end-to-end quality strategy for the Automation and Intelligence Pillar pod's AI products, such as Ivy, from test architecture through release readiness. This involves designing and building evaluation frameworks for non-deterministic systems. The focus areas include LLM output quality, agent behavior, regression detection, and guardrail or safety testing. You will develop reliable, scalable automated test suites and tooling. These will cover functional, system, API, integration, UI, and regression testing. The tooling will be integrated into the company's CI/CD pipeline.

Establishing quality metrics and benchmarks will be a core responsibility. Metrics include accuracy, latency, cost, hallucination rates, and failure rates. These benchmarks provide the team with confidence to ship and inform data-driven decisions. You will partner closely with Engineering Managers, Product Managers, Designers, ML and Data Science teams, and other engineering groups. This partnership ensures alignment on requirements, priorities, and definitions of done.

The role requires acting as a technical mentor and force multiplier across the QE organization. You will promote QE best practices and drive process improvements. This will elevate testing standards for AI systems across the company. Participation in technical planning, code review, and architectural decision-making is essential. You will advocate for testability and reliability from the design stage. Providing timely, actionable test result feedback to the Automation and Intelligence Pillar pod, the QA team, and other stakeholders is also required.

What You'll Bring to the Table

The position requires 7+ years of professional, hands-on software quality and test automation experience. Candidates must have a track record of setting technical direction on complex systems. Deep, hands-on experience with API and UI test automation frameworks is necessary. Examples include Playwright, Cypress, Jest, or Junit.

Fluency in at least one of the following languages is required: TypeScript, JavaScript, Python, or Java. Preference is given to TypeScript. Experience testing AI and agentic systems is essential. This includes working with evals, prompt regression testing, and output validation. Experience using AI to scale automation, such as test generation and self-healing tests, is required. Familiarity with technologies like OpenAI or Anthropic is necessary.

Experience with CI/CD tools such as Jenkins, Github Actions, or CircleCI is required. A strong understanding of scalable system design for web-based APIs and backend processing systems is mandatory. In-depth understanding of QA methodologies in a software company with fast release cycles is expected.

You must have a proven ability to define test case design standards and coverage strategy. This includes edge cases, negative paths, compatibility, performance, and concurrency scenarios. Ownership of a test case management practice, such as TestRail, is required. This practice should cover areas that others build upon.

Knowledge of React is important. This includes adding data-testids and other testing considerations. The ability to explore unfamiliar codebases to map features and quality risks quickly is required.

Compensation and Benefits

The compensation for this role ranges from $130,000 to $170,000 USD per year. The position is full-time. StackAdapt is a remote-first company open to candidates in Canada and the United States.

Nice to Have

Experience with large language models, agent evaluation, and AI-assisted test generation or self-healing tests is a plus.

Skills & Tools

StackAdapt, Ivy, Automation and Intelligence Pillar, Playwright, Cypress, Jest, Junit, TypeScript, JavaScript, Python, Java, Jenkins, Github Actions, CircleCI, TestRail, React.