LLM AI Quality Analyst (German) – Remote

About the role

This short‑term contract position focuses on evaluating and improving personalized AI responses for German‑language users. You will work remotely, dedicating about 20 hours per week while ensuring at least two hours of overlap with Pacific Standard Time.

Key responsibilities

• Design and execute multi‑turn conversational prompts (typically 1‑5 turns) based on personal context.
• Evaluate AI responses for relevance, grounding, integration of personal data, and overall helpfulness.
• Assess correct and incorrect use of personal data in model outputs.
• Perform side‑by‑side (SxS) evaluation and ranking of AI responses.
• Identify grounding errors, poor inferences, hallucinations, and forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Required profile

• High proficiency in German (reading and writing).
• Experience in data annotation, AI quality evaluation, content moderation, or related fields.
• Strong analytical thinking and attention to detail.
• Ability to evaluate nuanced and ambiguous AI responses.
• Experience in prompt design and understanding of personalization concepts.
• Comfortable using a primary personal Google account with enabled data sources.
• BS/BA degree or equivalent experience in a relevant analytical field.
• Excellent written communication and structured feedback skills.
• Self‑motivated, able to work independently in a remote setting.
• Reliable desktop/laptop with stable internet connection.

Required skills

What we offer

• Hourly compensation of $15.3.
• Flexible remote work environment.
• Opportunity to contribute to cutting‑edge AI personalization technology.