Overview
The Data Lead will play a pivotal role in the rebuilding of a large trading organization's enterprise data platform, focusing on integrating streaming, storage, transformation, and governance into a unified system. This hands-on position involves guiding the development of event-streaming pipelines and production data pipelines using a modern lakehouse architecture, while also mentoring and setting technical standards for other engineers across teams.
Responsibilities
- Lead the development of a modern data platform, ensuring high-quality software delivery.
- Guide the creation and optimization of event-streaming and production data pipelines.
- Implement standards and best practices for data engineering teams.
- Mentor engineers and provide technical direction throughout projects.
- Collaborate with cross-functional teams to integrate platform components.
Requirements
- Proven experience in leading the build or modernization of production data platforms.
- Strong hands-on expertise with a major streaming technology such as Kafka.
- Production experience with a lakehouse architecture and Spark-based analytics.
- Proficiency in Python and SQL, with experience in a major public cloud environment.
- Ability to mentor engineers and define technical standards for teams.
- Familiarity with technologies like Flink, dbt, or Iceberg is a bonus.
- Experience in trading or financial services is advantageous.