Overview
The Senior Data Engineer will collaborate with a consultancy's multidisciplinary team to expand their data engineering capabilities for a large-scale cloud data platform. This role focuses on designing and developing modern data solutions to effectively handle complex and high-volume datasets, particularly those relating to geospatial and mapping applications. The position emphasizes involvement throughout the entire data lifecycle, contributing to both architectural decisions and engineering best practices.
Responsibilities
- Design and develop modern data solutions for high-volume datasets.
- Collaborate with a multidisciplinary team to enhance cloud-based data platforms.
- Ensure effective data collection, processing, governance, and consumption.
- Contribute to architectural decisions and establish engineering best practices.
- Process and analyze complex location-based datasets.
- Build and support automated data pipelines.
Requirements
- Proven experience delivering data engineering solutions in Azure.
- Strong hands-on experience with Azure Databricks.
- Familiarity with geospatial, mapping, or location intelligence datasets.
- Experience with large-scale distributed data processing workloads.
- Proficiency in Python and SQL development.
- Experience with Spark-based technologies.
- Knowledge of modern cloud data platform architectures and patterns.
- Familiarity with CI/CD practices and modern software engineering principles.