Data Engineer who can develop, test, run, maintain, and migrate data pipelines.
Essential skills include Azure Databricks, Azure Data Factory, GitHub Actions CI/CD, building APIs with Python FastAPI, and consuming APIs with requests.
Terraform skills are a plus.
Years – - years of development experience in data engineering skills
Strong experience in Spark.
Understand complex data system by working closely with engineering and product teams.
Develop scalable and maintainable applications to extract, transform, and load data in various formats to SQL Server, Hadoop Data Lake or other data storage locations.