Overview
We are seeking an experienced Lead Data Engineer for a 3-5 month contract role focused on delivering a Databricks migration and overarching data platform transformation. In this hands-on position, you will collaborate closely with Data Engineers and Data Scientists, providing technical leadership while overseeing the design and optimization of scalable data solutions. This role also includes mentoring responsibilities and requires working in a hybrid model with 2-3 days per week in the office.
Responsibilities
- Lead the technical delivery of Databricks migration and data platform transformation.
- Design, build, and optimize scalable data pipelines using Databricks, Spark, Python, and SQL.
- Provide technical leadership and mentorship to Data Engineers and technical stakeholders.
- Establish engineering best practices across the data engineering team.
- Collaborate with Data Scientists and stakeholders to meet project requirements and objectives.
- Drive improvements in data engineering practices and platform performance.
- Support wider transformation initiatives within the organization.
Requirements
- Proven experience with Databricks and Spark.
- Strong proficiency in SQL and Python.
- Experience building modern cloud-based data platforms.
- Solid understanding of data modeling and engineering best practices.
- Demonstrated success in delivering migration or transformation projects.
- Comfortable providing technical leadership and mentoring roles.
- Experience working in Agile environments.