Overview
The Senior Data Engineer will be instrumental in creating and maintaining robust data pipelines and Lakehouse architectures. This role involves close collaboration with internal teams and external partners to enhance data systems, with a specific aim to boost fan engagement on digital platforms. The engineer will focus on designing effective ETL/ELT processes and optimizing data workflows.
Responsibilities
- Design and develop ETL/ELT pipelines in Azure and Databricks.
- Construct Kimball-style dimensional models for analytics and reporting.
- Implement automated testing for data quality assurance.
- Ensure compliance with data governance and regulatory standards.
- Collaborate with the wider Data team to optimize data pipelines.
- Enhance platform capabilities and performance.
Requirements
- Hands-on expertise with Databricks, PySpark, and Delta Lake.
- Proven ability to build production-grade ETL/ELT pipelines with API integration.
- Strong knowledge of Kimball methodology within Lakehouse frameworks.
- Advanced proficiency in Azure data services (ADF, ADLS Gen2, Event Hubs) and SQL.
- Excellent analytical and troubleshooting skills for data processing.
- Strong communication skills for stakeholder collaboration.