Design and build Spark data ETL pipelines on AWS data platform.
Collaborate with cross functional teams such as data scientists, fraud, marketing and other business stakeholders to understand their data needs and deliver reliable solutions.
Optimize data infrastructure – design and maintain robust data infrastructure by using modern data platform architecture.
Ensure data quality and reliability.
Innovate and follow best practices.
Ensure operational excellence of the data platform, including monitoring, incident response, performance optimization, and continuous improvement.
Quem estamos procurando:
Professional experience working in data warehousing, data architecture, and/or data engineering environments, especially using Spark, Hadoop, Hive, etc. with a solid understanding of streaming pipelines...