Senior Quality Engineer, AI and Automation

About the role

C3 Ai seeks a Senior Quality Engineer, AI and Automation to define and drive quality strategy for next-generation AI-enabled products and platforms. This role designs and builds scalable automated test frameworks, develops novel approaches for validating AI-generated outputs and agentic workflows, and establishes metrics that ensure enterprise-grade reliability. You will operate at the intersection of software quality engineering and applied AI, embedding quality into every stage of the development lifecycle. This is an opportunity for an experienced quality engineer passionate about AI and automation to operate with high autonomy on complex, high-visibility initiatives. You will influence quality strategy across multiple teams, provide technical leadership and mentorship, and partner closely with software engineering, product, and AI stakeholders to elevate how enterprise AI is tested and delivered.

Key facts

What you'll do

• Define and execute quality strategies for AI-enabled products and platforms while ensuring alignment with enterprise reliability standards.
• Design, develop, and maintain scalable, automated test frameworks that span UI, API, and end-to-end testing for complex distributed systems.
• Create comprehensive test plans that cover functional, integration, end-to-end, performance, and reliability testing to validate system behavior under diverse conditions.
• Develop automated validation approaches for AI-generated outputs and agentic workflows, assessing accuracy, reliability, consistency, and safety through structured evaluation methods.
• Establish quality metrics and dashboards that provide visibility into product health and AI system effectiveness, enabling data-driven decisions.
• Integrate automated tests into CI/CD pipelines and build automated quality gates for AI workflows to enforce release readiness and reduce regression risk.
• Identify quality risks early in the development lifecycle and drive resolution strategies across complex, interdependent systems and components.
• Evaluate and implement AI-powered testing tools and methodologies to enhance test efficiency, coverage, and insight into system behavior.
• Mentor engineers on quality practices and promote quality engineering best practices, driving continuous improvement initiatives across the organization.
• Collaborate with cross-functional teams including software engineering, product, and AI to ensure quality considerations are embedded in design and implementation decisions.
• Contribute to the creation and refinement of test artifacts, including test cases, test scripts, and traceability matrices for audit and compliance needs.
• Support the development and execution of test environments and test data strategies that enable reliable and repeatable validation of AI features.
• Analyze test results and system telemetry to diagnose issues, isolate root causes, and provide actionable feedback to engineering teams for remediation.
• Champion the adoption of modern testing standards and tools, ensuring that testing practices evolve with emerging AI and automation technologies.

Requirements

• Hold a Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or a related field.
• Bring 5 or more years of experience in Quality Engineering, Test Automation, SDET, or Software Engineering roles.
• Demonstrate strong programming skills in Java, Python, or a similar object-oriented language.
• Have hands-on experience developing and maintaining automated test frameworks with UI, API, and end-to-end automation testing.
• Be fluent with modern test automation tools such as Playwright, Selenium, Cypress, or equivalent.
• Have integrated automated tests into CI/CD pipelines and worked with Git-based version control.
• Possess a deep understanding of the software development lifecycle, Agile methodologies, and quality engineering best practices.
• Demonstrate the ability to analyze complex systems, architect test frameworks, and identify quality risks early in the development process.
• Show commitment to writing clean, maintainable test code and collaborating effectively with engineering teams in a fast-paced environment.
• Exhibit strong analytical and problem-solving skills, with the ability to translate ambiguous requirements into concrete test strategies and coverage plans.
• Communicate clearly and professionally in written and verbal English, enabling effective collaboration with global stakeholders and team members.

Nice to have

• Experience testing AI-powered applications, machine learning systems, LLMs, or AI agents in production or near-production environments.
• Experience designing evaluation strategies for AI outputs, including accuracy, reliability, consistency, and safety measurements.
• Familiarity with AI evaluation frameworks such as DeepEval, LangSmith, Ragas, or similar tools to standardize assessment practices.
• Experience validating Retrieval-Augmented Generation systems and knowledge of prompt engineering and AI model evaluation techniques.
• Experience building automated quality gates for AI workflows and AI-assisted test generation to streamline testing and improve coverage.

Practical notes

-

Location: Guadalajara, Jalisco, Mexico.

-

Engagement: Full-time.

-

Compensation: $70,000.00 USD per year.

• Skills & Tools: C3 Ai, C3 Agentic AI Platform, C3 Ai applications, Java, Python, Playwright, Selenium, Cypress, Git, CI/CD.