Overview
The Senior Data Engineer will play a crucial role in a high-impact team dedicated to rebuilding and optimizing data pipelines within a Databricks Lakehouse environment. Working closely with stakeholders, the contractor will focus on delivering clean, scalable, and high-quality data for analytics and reporting. This position requires on-site work one day per week and provides an opportunity to work on a greenfield project.
Responsibilities
- Design, build, and optimise data pipelines using ELT/ETL best practices.
- Migrate and transform data into a Databricks Lakehouse architecture.
- Ensure data quality, reliability, and scalability for analytics and reporting.
- Collaborate with stakeholders to deliver robust solutions aligned with business needs.
- Support production environments and troubleshoot performance issues.
Requirements
- 7+ years of experience in Data Engineering with strong expertise in cloud-based data platforms.
- Proven ability to design and optimise pipelines for large-scale data processing.
- Hands-on experience with Databricks and Azure.
- Strong stakeholder communication and problem-solving skills.
- Proficiency in Databricks, DBT, Python, PySpark, SQL, and Azure.
- Experience in eCommerce environments is a bonus.