Overview
We are seeking a Senior Data Engineer for a 6-month contract, focused on enhancing the data architecture for a global investment platform. This role will involve collaborating with various stakeholders to reconcile irregular investment data and develop a robust, scalable data layer. The ideal candidate will utilize their expertise in Python, Azure, and data ingestion to deliver high-quality solutions.
Responsibilities
- Build and evolve data ingestion pipelines from multiple external data sources.
- Design scalable data models across complex investment data.
- Build APIs and product-facing data services.
- Solve reconciliation and data-quality problems at scale.
- Develop lakehouse and streaming capabilities.
- Improve reliability, observability, and performance.
- Develop AI-driven approaches to ingestion, processing, and reconciliation.
Requirements
- Proven experience as a Data Engineer using Python and TypeScript.
- Strong proficiency with Azure Data Lake and Airflow.
- Hands-on experience in designing and implementing scalable data models.
- Familiarity with APIs and data services development.
- Experience with infrastructure tools like AKS, Azure DevOps, and Terraform.
- Knowledge of databases and workflows related to AI processes.
- Ability to solve complex data challenges in real-world scenarios.