Overview
The Technical Lead will oversee a high-performing team of Data Engineers, guiding them through the complexities of data architecture and pipeline implementation. Working in a collaborative environment, this role will focus on leveraging the Databricks platform while serving as the primary point of contact for cross-functional stakeholders. The Technical Lead’s expertise in Python, Azure, and data engineering practices will be essential in driving project success and ensuring technical excellence across the team.
Responsibilities
- Direct and manage a team of 20 Data Engineers, fostering a collaborative environment.
- Conduct code reviews and provide mentorship to ensure code quality and adherence to best practices.
- Architect and implement scalable data pipelines and architecture using Python and Databricks.
- Act as the primary Subject Matter Expert for the Databricks platform, guiding the team through technical challenges.
- Communicate complex business requirements to actionable technical specifications.
- Plan and manage the team's workload to ensure timely delivery of data initiatives.
- Evaluate and optimize data infrastructure for performance, reliability, and security.
- Interface with cross-functional teams to align on project goals and timelines.
Requirements
- Extensive experience with the Databricks platform, including Apache Spark and Delta Lake.
- Strong proficiency in Python and Azure technologies.
- Demonstrated ability to lead and mentor technical teams effectively.
- Experience in architecting scalable and robust data pipelines.
- Strong analytical and problem-solving skills, with the ability to make impactful architectural decisions.
- Familiarity with CI/CD practices in software engineering.
- Excellent communication skills, both technical and non-technical.
- Ability to manage multiple priorities and stakeholders effectively.