Lead Data Engineer

Overview

The Lead Data Engineer will spearhead a team of 5 to 7 engineers, guiding the transition of data products into a new governed Databricks workspace. This role balances leadership responsibilities with hands-on engineering tasks, ensuring collaboration between data teams and business stakeholders to deliver quality data products that cater to various operational needs.

Responsibilities

  • Lead and coach a team of data engineers, setting technical direction and driving delivery.
  • Collaborate with business stakeholders and data product owners to prioritize and define project requirements.
  • Engage in hands-on data modelling, pipeline design, and code reviews to ensure high standards of quality.
  • Develop reusable Databricks templates and libraries to enhance data product efficiency.
  • Prepare data products for integration with AI tools such as Databricks Genie.
  • Establish standards for testing, CI/CD, and data quality across the team.

Requirements

  • Minimum of 7 years of experience in data engineering, including at least 2 years in a leadership or mentoring role.
  • Strong hands-on expertise with Databricks, Spark or PySpark, Python, and SQL.
  • Demonstrated ability to translate business priorities into technical solutions.
  • Proficient in data modeling and building production data products.
  • Experience working with Unity Catalog, Delta Lake, and Delta Live Tables is essential.
  • Fluent in English; proficiency in French is a plus but not required.