AI Quality & Validation Specialist

Responsibilities

• Support functional, regression, and user-focused testing of applications and AI-enabled features before release
• Evaluate AI-generated outputs for quality, relevance, accuracy, consistency, and alignment to prompts, instructions, or reference materials
• Identify, document, and track software defects, content issues, edge cases, and output anomalies
• Perform scenario-based testing by interacting with systems as an end user to validate workflows and expected outcomes
• Flag issues in AI or application behavior and provide clear written feedback on what should be corrected or improved
• Assist with validation of prompts, workflows, procedures, and output-review criteria used in AI-supported processes
• Help review text, image, audio, video, or other generated outputs, as applicable to the program
• Maintain test evidence, validation notes, issue logs, and supporting documentation in accordance with project standards
• Collaborate with developers, analysts, engineers, and project leadership to clarify expected behavior and support issue resolution
• Participate in pilot testing of new features, experimental capabilities, and process improvements
• Support continuous quality improvement efforts by identifying recurring issues, trends, and opportunities for better testing coverage
• Contribute to the development of repeatable validation procedures, checklists, and quality review practices

Required Qualifications

• 1–3 years of relevant experience in software testing, QA support, AI output evaluation, digital content review, or an Associate’s / Bachelor’s Degree in a related field
• Strong attention to detail with the ability to identify inconsistencies, defects, or quality issues
• Experience documenting findings clearly and communicating actionable feedback
• Ability to follow structured instructions, evaluation criteria, and testing procedures
• Strong written and verbal communication skills
• Comfortable working across multiple tasks in a fast-paced environment
• Ability to think from the end-user perspective and test systems against expected outcomes
• Basic familiarity with modern software applications, web-based systems, and common collaboration/documentation tools

Desired Qualifications

• Experience evaluating generative AI outputs such as text, images, audio, or video
• Experience supporting software QA, test execution, UAT, or validation activities
• Familiarity with defect tracking tools, test case management, or Agile delivery environments
• Experience comparing outputs against prompts, business rules, or reference materials
• Exposure to prompt testing, AI workflow review, or human-in-the-loop evaluation processes
• Experience in a federal contracting environment
• Active clearance or eligibility to obtain required clearance, if applicable