Overview
The Engineering Manager for the Data Platform will oversee the development of a cutting-edge data infrastructure on Databricks and AWS. This role involves leading two engineering teams and collaborating with external partners to enhance data processing capabilities, ensuring that data delivery is significantly expedited. The manager will play a critical role in implementing a streaming backbone and enabling change data capture from trading systems to deliver timely data insights.
Responsibilities
- Lead engineering teams to develop and implement the data platform infrastructure.
- Deliver a streaming backbone with Kafka and change data capture capabilities.
- Collaborate with external partners to enhance platform functionalities.
- Set strategic agendas and drive implementation across multiple teams.
- Leverage deep expertise in Databricks and AWS for project execution.
- Manage production operations involving data streaming technologies and tools.
Requirements
- 10+ years of experience in data/platform engineering.
- 4+ years of experience in managing engineering teams.
- Proven production experience with Kafka, CDC technologies (Debezium, GoldenGate, DMS), and Spark Streaming or Flink.
- Strong expertise in Databricks and AWS platforms.
- Demonstrated ability to lead vendors and systems integrators effectively.
- Confident in setting agendas and implementing across multiple teams.