Overview
We are seeking a Senior Python / PySpark Data Engineer with expertise in Azure Databricks to join a dynamic, remote team. In this role, you will collaborate with other engineers to design, develop, and optimize scalable data solutions for enterprise-scale cloud platforms. This position is ideal for someone who thrives on solving technical challenges and delivering high-quality, production-ready code.
Responsibilities
- Design and develop data solutions using Azure Databricks.
- Build scalable data pipelines with Python and PySpark.
- Develop batch and streaming data processing solutions.
- Analyse complex datasets to create efficient data models.
- Deliver robust ETL/ELT pipelines ready for production.
- Collaborate with engineering teams to create high-quality cloud data solutions.
Requirements
- Strong experience with Azure Databricks.
- Proficient in Python and PySpark.
- Solid background in SQL and core Data Engineering principles.
- Experience in building enterprise-scale data pipelines.
- Knowledge of batch and streaming data processing.
- Familiarity with Unity Catalog and Databricks functionalities.
- Experience with Azure DevOps and CI/CD pipelines is a plus.