Overview
In this role as a Databricks Data Engineer, you will be responsible for leading the development of a modern data platform for a large financial services organization. Collaborating with both onshore and offshore teams, you will focus on transitioning from batch-based analytics to a streaming-driven architecture, ensuring alignment with senior stakeholders throughout the process.
Responsibilities
- Define an end-to-end data vision covering current state and target architecture.
- Design and develop a Databricks-based medallion architecture with a focus on analytics readiness.
- Improve data availability by transitioning to more frequent or streaming ingestion.
- Lead technical decision-making on data use cases and prioritisation.
- Collaborate with onshore and offshore teams to resolve blockers and streamline delivery.
- Provide regular updates to senior stakeholders to maintain alignment on project progress.
Requirements
- Strong experience as a Lead Data Engineer in financial services, preferably in pensions.
- Proven ability in designing modern data platforms using Databricks and medallion architecture.
- Experience integrating large legacy data volumes with contemporary analytics solutions.
- Proficient in translating business needs into technical implementation.
- Demonstrated ability to lead teams in both onshore and offshore settings.
- Excellent communication skills to engage effectively with senior stakeholders.