AI tester with Arabic language

About the role

This initiative represents a critical evaluation phase in the development lifecycle of an artificial intelligence system, specifically targeting the robustness of its safety protocols. You will operate as a specialized evaluator, engaging in direct dialogue with the system to test its boundaries and resilience. A core component of your mandate will involve formulating targeted prompts derived from predefined risk categories where training data exists. The essential objective is to systematically elicit responses that may be unsafe, including content that is harmful, offensive, dangerous, or toxic. Your analytical work will provide the necessary evidence to refine guardrails and ensure that unsafe content is effectively mitigated before public release. Success in this role will be defined by the measurable improvement in the security and reliability of the AI deployment pipeline. The position is structured as a fully remote, full-time engagement with an initial flexible timeline that may extend based on the complexity and progress of the evaluation. Furthermore, comprehensive online onboarding and standardized training protocols will be delivered to all selected candidates prior to any live testing activities.

Key facts

What you'll do

• Design and execute a structured testing protocol for the AI agent, adhering strictly to the project scope and testing mandate provided by the initiative.
• Formulate a diverse range of input sequences and conversational prompts intended to probe the model's limitations across multiple predefined categories of risk.
• Conduct systematic examinations of model replies to identify recurring patterns of dangerous, toxic, or otherwise unsafe language generation within the system outputs.
• Redirect active conversation threads to rigorously verify the model's ability to refuse unsafe requests and maintain adherence to safety guidelines during live interaction.
• Construct intricate evaluation scenarios that challenge the AI agent's capacity to interpret instructions while balancing nuanced linguistic and contextual variables.
• Manage the intake and processing of prompts within the testing environment to ensure a controlled and traceable evaluation workflow.
• Coordinate closely with engineering and safety teams through coordinated review cycles to ensure findings are documented with clarity and precision.
• Analyze the results of each testing iteration to shape concrete shipping criteria that determine when the agent is sufficiently safer for deployment.
• Establish and participate in partner aligned checks that ensure feedback integrates seamlessly into future training iterations and model improvements.
• Maintain detailed, organized logs that track every prompt issued and the corresponding model answer to guarantee full traceability and auditability.
• Assess the effectiveness of the current guardrails in resisting pressure and preventing the generation of undesirable content.
• Provide actionable feedback regarding the stability and reliability of the AI system under adversarial conditions.
• Collaborate with internal stakeholders to prioritize testing areas that present the highest risk to end users.
• Contribute to the definition of practical metrics that quantify the success of the safety improvements implemented throughout the project.

Requirements

• Hold advanced Arabic language skills that enable fluent conversation and a precise understanding of nuanced meaning, as this linguistic capability is strictly non-negotiable for the successful execution of the role.
• Possess a mandatory history of three to five years of prior experience acting as an AI tester within similar evaluation positions.
• Demonstrate availability for a full-time schedule over several consecutive months, which is essential for maintaining the continuity and momentum of the evaluation process.
• Ensure reliable access to a stable internet connection, as this is mandatory for all project activities and communication channels.
• Commit to the completion of all provided training modules before active testing commences, ensuring full comprehension of procedures and safety benchmarks.
• Exhibit strict adherence to protocol and methodology once training is concluded, following all guidelines without deviation.
• Maintain a high level of professionalism and discretion when handling sensitive or potentially harmful content during evaluation sessions.
• Possess strong analytical skills to interpret model behavior and categorize responses based on safety criteria.

Nice to have

No specific "nice to have" qualifications are outlined at this stage; the evaluation relies strictly on the core skills and experience detailed above.

Practical notes

The engagement is a contract full-time position based in Cairo. The project timelines remain flexible and may extend based on evaluation progress and complexity.