Speech AI Evaluation Specialist

About the role

This role represents a specific opportunity dedicated to the rigorous assessment of AI-generated Vietnamese content within a United States operational framework. As a Speech AI Evaluation Specialist, you will actively influence the development and safety of contemporary AI models through meticulous analysis. This position operates as a freelance engagement, designed for part-time participation exceeding ten hours weekly, with immediate availability and a flexible work-from-home schedule. The duration remains to be confirmed, and compensation is set at eighteen United States dollars per hour. The primary purpose of this role is to ensure the quality and reliability of speech AI outputs concerning the Vietnamese language. Individuals who are students, recent graduates, stay-at-home parents, gig workers, or professionals seeking adaptable remote work are encouraged to apply. Your contributions will help shape the future direction of technology by providing essential human judgment.

Your core responsibilities involve direct interaction with AI systems during short voice conversations to evaluate their performance. You will utilize designated platforms and adhere to specific scenarios and prompts provided for each session to maintain consistency. The evaluation process requires you to assess AI responses based on relevance, accuracy, clarity, and overall quality using established criteria. You will provide objective ratings and written feedback in strict accordance with project guidelines to ensure data integrity. It is essential to complete all assigned tasks while meeting established standards for both quality and productivity to support the project's goals.

We seek candidates with specific capabilities that form the foundation for success in this position. Native-level fluency in Vietnamese is mandatory to conduct correct and effective evaluations of spoken language. Also, you must demonstrate fluent or advanced English proficiency, corresponding to levels B2 through C2 on the Common European Framework of Reference for Languages, to understand instructions and communicate efficiently. Preferred experience includes working in areas such as machine learning tasks, data collection and preprocessing, data evaluation and quality assurance, and data annotation and labeling. Possessing background knowledge in these domains allows for a more precise contribution to the project goals and a deeper understanding of the evaluation context.

This position offers several advantages that cater to a diverse range of professionals. You will enjoy a flexible schedule that accommodates other commitments, such as education or caregiving responsibilities. The role provides an opportunity to earn additional income through timely payments on a contractual basis. The structure is particularly suitable for students, part-time workers, or stay-at-home parents seeking meaningful remote work. Individuals selected for this position will be required to follow a new duty order that defines evaluation protocols and ensures standardization across all assessments.

Your daily work will involve translating project objectives into practical rating actions that directly impact AI development. You will set up test scripts specifically for Vietnamese language evaluation sessions to ensure comprehensive coverage of use cases. This includes initiating voice trials with AI systems by using provided scenario lists to simulate real-world interactions. During these trials, you will measure the precision of answers and the natural flow of conversation to identify any deficiencies or biases. Assigning scores that reflect established guidelines without personal bias is a critical task that maintains the objectivity of the evaluation. You will also track your weekly output to ensure it meets productivity standards and aligns with project expectations.

A significant portion of the evaluation focuses on reviewing conversation transcripts to extract actionable insights. This activity helps identify recurring understanding gaps within the system that may hinder performance. You will document edge cases that standard prompts fail to capture effectively, providing valuable data for system improvement. Coordination with engineers is necessary to refine annotation instructions and ensure their accuracy for future evaluations. This collaborative effort ensures that data evaluation maintains a high standard of quality and contributes to the overall success of the AI system.

The requirements for this role are clear and serve as a hard barrier to entry. You must speak Vietnamese at a native level to perform correct evaluations of nuanced language and cultural context. English communication must be handled with fluency or advanced proficiency to understand complex instructions and provide clear feedback. Experience in machine learning tasks or data annotation is considered a background requirement for candidates entering this role. No additional skills or tools are specified in the current documentation, keeping the focus on linguistic and evaluative abilities. It is important to note that eligibility is strictly defined by these criteria.

Please This document serves as a new duty order replacing the previous draft and supersedes any prior information. It outlines the responsibilities and expectations for the Speech AI Evaluation Specialist position at Rws in a definitive manner. All applicants are encouraged to review the official sources for the most current information regarding compensation, scheduling, and procedural requirements.

What you'll do

• Utilize specific scenarios and prompts provided to guide your assessment process and maintain consistency.
• Assess AI responses based on relevance, accuracy, clarity, and overall quality using established evaluation criteria.
• Provide objective ratings and written feedback for each interaction in accordance with project guidelines.
• Complete all assigned tasks while meeting established standards for both quality and productivity.
• Set up test scripts specifically for Vietnamese language evaluation sessions to prepare for testing.
• Initiate voice trials with AI systems by using provided scenario lists to simulate realistic interactions.
• Measure the precision of answers and the natural flow of conversation during these trials.
• Assign scores that reflect established guidelines without personal bias to ensure fair assessment.
• Track your weekly output to verify compliance with productivity standards and project requirements.
• Review conversation transcripts to identify recurring understanding gaps within the AI system.
• Document edge cases that standard prompts fail to capture effectively for further analysis.
• Coordinate with engineers to refine annotation instructions and ensure their accuracy for future use.
• Collaborate with team members to maintain a high standard of quality in data evaluation.
• Translate project objectives into practical rating actions that influence AI model development.
• Contribute to the identification of systemic issues through detailed analysis of transcripts and feedback.
• Support the overall goal of improving AI safety and performance for Vietnamese language applications.
• Engage in continuous learning regarding evaluation protocols to enhance the accuracy of your assessments.

Requirements

You must meet the following eligibility and hard bars to be considered for this role. Native-level fluency in Vietnamese is mandatory to perform correct and effective evaluations of spoken language. You must also demonstrate fluent or advanced English proficiency, corresponding to levels B2 through C2 on the Common European Framework of Reference for Languages. Experience in machine learning tasks or data annotation is considered a background requirement for this position. No additional skills or tools are specified in the current documentation, ensuring the role remains focused on linguistic capability. You are expected to adhere strictly to the project guidelines and quality standards throughout your engagement. This role is not suitable for individuals who cannot meet these specific language and experience criteria.

Practical notes

Hours: This is a freelance engagement, designed for part-time participation exceeding ten hours weekly.

Travel: Not applicable, as the role is fully remote.

in the provided source information.

Deadlines: Immediate commencement is available for this position.