Data Engineer – Web Scraping

About the role

This position is based in Luxembourg and focuses on building scalable web‑scraping solutions and data pipelines that feed business‑critical insights. You will collaborate with analysts, engineers, and cross‑functional teams to deliver reliable datasets that support strategic decision‑making.

Key responsibilities

• Design, develop and maintain automated web scrapers for structured and unstructured sources.
• Clean, transform, validate and manipulate large datasets using Python and Pandas.
• Build and maintain data ingestion pipelines into databases or data warehouses.
• Schedule, monitor and optimise scraping workflows with Apache Airflow.
• Implement quality‑control checks to ensure data integrity and availability.
• Troubleshoot pipeline issues and resolve time‑sensitive production incidents.
• Enhance internal tools, automation frameworks and platform capabilities.
• Work with engineering teams to create scalable, maintainable data processing workflows.

Required profile

• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• 2‑4 years of professional software development experience.
• Strong problem‑solving mindset and proactive attitude.

Required skills

• Python programming
• SQL and database knowledge
• Pandas library
• HTML, JavaScript, APIs
• Web‑scraping tools such as Selenium, Scrapy, XPath, Fiddler, Postman
• Workflow orchestration with Apache Airflow