Overview
We are seeking experienced Software Engineers for an initial 6-month contract position focused on enhancing a data platform supporting a global investment product. The successful candidates will join a collaborative engineering team, tackling the challenges of fragmented investment data by designing and implementing robust data pipelines, models, and APIs. This role offers the opportunity to work with cutting-edge technologies in a pragmatic environment, aimed at improving data quality and supporting AI/ML initiatives.
Responsibilities
- Build and orchestrate ingestion pipelines for complex structured and unstructured data.
- Design scalable data models across various architectural formats including relational, NoSQL, blob, and Delta.
- Transform and reconcile inconsistent data from multiple external sources.
- Develop APIs and data services based on the refined data.
- Enhance data quality, lineage, observability, and reliability through systematic investigation of data issues.
- Implement long-term fixes for identified data problems.
- Contribute to the development of AI/ML capabilities in data ingestion and reporting.
Requirements
- Proven experience in building production data pipelines and data models.
- Strong programming skills in TypeScript and/or Python.
- Familiarity with technologies such as Airflow, Delta Lake, Azure Data Lake, and relational & NoSQL databases.
- Experience with cloud environments, particularly Azure, along with tools like AKS and Docker.
- A solid understanding of data processing and observability standards.