Evaluate and define excellence in real-world digital product design by creating task-specific grading criteria and scoring completed design work. You will not produce design deliverables yourself, instead you will set the bar for quality and write rigorous justifications for every score to enable reproducible, defensible evaluation of both AI-generated and human work.

• Design precise, task-specific grading rubrics for digital product and UX deliverables, including design systems and component libraries, responsive UI layouts and dashboards, end-to-end product flows such as onboarding, checkout, and booking, logged-in customer portals, UX writing and product content, and design-to-engineering or production handoff.
• Score AI-generated and human work samples against those criteria, providing detailed written justifications for every score.
• Apply consistent, evidence-based judgment so that scores are reproducible and defensible.
• Receive and incorporate structured feedback from senior reviewers, iterating quickly to improve rubrics and scoring.

• 5+ years of professional experience in product design, UI/UX, or design systems for real digital products, not brand or marketing sites.
• Experience at leading digital-product or UX studios, in-house product-design teams at product-led companies, or e-commerce and marketplace companies with sophisticated customer portals and dashboards.
• Background across one or more relevant roles, for example Product / UI / Design Systems Designer, UX Lead, UX Writer / Content Designer, or digital Product / Business Analyst.
• Deep fluency in interaction and visual design, design systems, responsive layouts, end-to-end product flows, prototyping, and production handoff in Figma.
• Ability to articulate critique like a design or UX lead, with exceptionally strong written communication to explain exactly why work meets or does not meet the bar for real products and users.
• Detail-oriented and consistent, comfortable having your judgment reviewed and calibrated against peers.
• Prior experience with AI training, evaluation, or human-data projects is a strong plus.

• Location: Remote.
• Employment type: hourly engagement.

• Pay range: 80 - 150 hourly.

• To apply, submit your resume and portfolio or other relevant professional background materials.
• Qualified candidates may be asked to complete a brief assessment or provide additional information as part of the selection process.