We are seeking an experienced Lead Data Engineer with advanced expertise in PySpark and hands‑on experience building ETL pipelines, data lake architectures, and integrating data feeds on AWS.

You will handle both structured and unstructured data, ingesting information from a variety of on‑premises and enterprise sources such as SAP, Intelex, SQL, and OSI PI into AWS. This position provides the chance to work on large‑scale data projects and collaborate with diverse teams in a fast‑paced setting.

Responsibilities

• Create, refine, and manage ETL pipelines using PySpark and AWS Glue Jobs to process extensive structured and unstructured datasets
• Coordinate data workflows with Apache Airflow, ensuring dependable scheduling, dependency management, and effective error handling
• Develop and sustain data feeds from on‑premises and enterprise systems into AWS data lake environments
• Integrate with enterprise sources includi...