AI Evaluation Specialist – Italian (Remote)

About the role

We are looking for an Italian‑speaking AI Evaluation Specialist to assess the quality of large language model responses. The role is fully remote and offers a flexible 20‑hour‑per‑week schedule, allowing you to work alongside a global team focused on personalization, grounding and helpfulness of AI outputs.

Key responsibilities

• Evaluate AI model responses for personalization quality, including grounding, integration and helpfulness.
• Design and execute multi‑turn prompts that incorporate personal context to test model capabilities.
• Analyze outputs for hallucinations, incorrect personalization and flawed inferences.
• Perform side‑by‑side comparisons of model outputs to determine relative quality.
• Write clear, structured rationales and rankings for each evaluation.
• Provide detailed feedback and annotations to improve model performance.
• Maintain strict data hygiene and accurate documentation of all evaluations.

Required profile

• Strong proficiency in Italian with excellent reading and writing skills.
• Experience in data annotation, AI evaluation, content moderation or similar analytical roles.
• Analytical mindset capable of evaluating nuanced AI responses.
• Ability to design creative multi‑turn prompts based on personal context.
• Excellent written communication and attention to detail.
• Full‑time availability with flexibility to overlap with PST time zones.
• Bachelor’s degree or equivalent experience in a relevant analytical field.

Required skills

• Italian language proficiency (reading & writing).

What we offer

• Hourly compensation ranging from $11.5 to $15.3 USD.
• Remote work with a flexible 20‑hour weekly commitment.
• Opportunity to contribute to cutting‑edge AI personalization research.