Overview
As a Data Engineer, you will play a crucial role in a significant Records Management System (RMS) replacement programme within the public sector. Collaborating closely with solution architects and suppliers, you will design and build data pipelines, support data migration activities, and enhance reporting and analytics capabilities across a complex enterprise environment.
Responsibilities
- Design, develop, and optimise scalable ETL/ELT pipelines to support data migration, integration, and reporting requirements.
- Build data ingestion and transformation solutions across multiple source and target systems.
- Analyse existing datasets, schemas, and data structures to support mapping and migration activities.
- Collaborate with solution architects and suppliers to implement target data models and integration patterns.
- Drive data quality initiatives, including validation, cleansing, monitoring, and exception management.
- Support testing, trial migrations, cutover activities, and ensure production readiness throughout the programme lifecycle.
Requirements
- Strong commercial experience as a Data Engineer in large-scale transformation or migration programmes.
- Expert knowledge of ETL/ELT development, data integration, and scalable data pipeline design.
- Proven experience with complex data migration and transformation projects involving multiple source and target systems.
- Strong understanding of data modeling, relational databases, data warehousing, and data lake architectures.
- Hands-on experience with Microsoft Azure data services and cloud-based data platforms.
- Familiarity with technologies such as Azure Data Factory, Azure Synapse Analytics, Python, ETL tooling, and API integrations.
- Strong knowledge of data quality, governance, compliance, monitoring, and data integrity frameworks, particularly with large operational datasets.
- Active NPPV 3 Clearance or ability to obtain it, along with relevant experience in policing or public sector environments.