German LLM AI Quality Analyst (Remote, $15/hr)

About the role

We are looking for a German‑speaking specialist to evaluate and improve personalized AI responses generated by large language models. This short‑term remote contract focuses on ensuring relevance, grounding, and correct use of personal data in multi‑turn conversations.

Key responsibilities

• Design and execute multi‑turn conversational prompts (1‑5 turns) based on personal context.
• Evaluate AI responses for relevance, grounding, integration, and helpfulness.
• Assess correct and incorrect use of personal data in model outputs.
• Perform side‑by‑side (SxS) evaluation and ranking of AI responses.
• Identify grounding errors, hallucinations, poor inferences, and forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Required profile

• German fluency (reading and writing) with high proficiency.
• Experience in data annotation, AI quality evaluation, content moderation, or related roles.
• Strong analytical thinking and attention to detail.
• Ability to evaluate nuanced and ambiguous AI responses.
• Experience in prompt design and understanding of personalization concepts.
• Comfortable using a personal Google account with enabled data sources.
• BS/BA degree or equivalent experience in a relevant analytical field.
• Strong written communication and structured feedback skills.
• Self‑motivated, able to work independently in a remote setting.
• Reliable desktop/laptop with stable internet connection.

Required skills

• German language proficiency (reading & writing)
• Data annotation
• AI quality evaluation
• Prompt design
• Content moderation

What we offer

• Short‑term contract (1 month) with immediate start.
• Flexible remote work (20 hours/week, minimum 2 hours overlap with PST).
• Competitive rate of $15 per hour.