German LLM AI Quality Analyst (Remote)

About the role

We are seeking a German‑speaking LLM AI Quality Analyst to evaluate personalized conversational AI outputs. This short‑term, remote contract involves designing prompts, reviewing model responses, and ensuring data privacy compliance.

Key responsibilities

• Design multi‑turn conversational prompts (1‑5 turns) based on personal context.
• Evaluate AI responses for relevance, grounding, integration of personal data, and overall helpfulness.
• Identify and document correct and incorrect use of personal data in model outputs.
• Perform side‑by‑side (SxS) ranking of responses and flag grounding errors, hallucinations, or forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations after review.

Required profile

• Fluent in German (reading and writing) with high proficiency.
• Experience in data annotation, AI quality evaluation, content moderation, or related fields.
• Strong analytical thinking, attention to detail, and ability to assess nuanced AI outputs.
• Familiarity with prompt design and personalization concepts.
• BS/BA degree or equivalent experience in an analytical discipline.
• Excellent written communication and ability to provide structured feedback.
• Self‑motivated, able to work independently in a remote setting.
• Reliable computer with stable internet connection.

Required skills