Overview
We are looking for skilled Data Engineers to join our dynamic consultancy team, focusing on building robust systems for data management across various industries, including telecoms and finance. This role offers an opportunity to work with cutting-edge technologies in a flexible environment while collaborating with cross-functional teams to meet client needs and exceed expectations. Whether on a contract basis or as part of a longer-term opportunity, your expertise will be pivotal in our mission to enhance data engineering capabilities and support client challenges in the data world.
Responsibilities
- Develop and maintain data pipelines using Azure Data Factory (ADF).
- Transform and model data for downstream consumption using SQL and Python.
- Collaborate with teams to understand data requirements and implement solutions.
- Ensure data quality, integrity, and security throughout the data pipeline.
- Troubleshoot and resolve issues in the data pipelines.
- Stay updated with Azure technologies and best practices.
- Design and implement metadata-driven pipelines to automate processing tasks.
Requirements
- Solid experience with Azure, specifically Azure Databricks and Azure SQL/SQL Server.
- Proficiency in SQL and Python languages.
- Hands-on experience in building data pipelines using Azure Data Factory (ADF).
- Familiarity with metadata-driven pipelines and Azure storage.
- Knowledge of IT concepts, including security and networking.
- Experience with Azure DevOps pipelines is preferred.
- Bonus: Exposure to Apache Airflow, DBT, or other cloud platforms.