Overview
We are seeking a Lead Data Engineer to deliver a six-month contract role, focused on designing and implementing sophisticated data engineering solutions using Databricks. The successful contractor will work closely with engineers, architects, and business stakeholders to create efficient systems while maintaining a strong emphasis on engineering quality and continuous improvement. This role requires a hands-on approach and will involve being on-site five days a week in London due to specific project requirements.
Responsibilities
- Design, develop and deploy production-grade data engineering solutions using Databricks and dbt.
- Take hands-on responsibility for coding, implementation and deployment within relevant platforms.
- Suggest improvements to engineering practices covering CI/CD, testing, version control and deployment processes.
- Design and implement effective metadata management, monitoring, alerting and operational controls.
- Contribute to the architecture and development of scalable enterprise data solutions.
- Support the evolution of engineering standards, technical processes and operational practices.
- Identify potential architectural and delivery challenges, applying lessons learned from previous implementations.
- Advise engineers, architects and business stakeholders to translate requirements into effective technical solutions.
Requirements
- 5+ years of professional experience in data engineering or a closely related technical discipline.
- Strong hands-on experience developing and deploying solutions using Databricks.
- Hold the certification of Databricks Certified Data Engineer Professional.
- Proven practical experience with dbt, including coding and deployment.
- Strong AWS experience - essential.
- Solid understanding of data engineering principles, including CI/CD, testing, metadata management, monitoring and alerting.
- Experience designing and implementing data engineering solutions within complex technical environments.
- Experience establishing or improving engineering standards and operational practices.