Overview
Senior Data Engineer - Python, SQL, Airflow, Redshift, Snowflake I've partnered with a high-growth start-up looking to expand their Data function, by bringing on a Senior Data Engineer on a contract basis - more info below: Senior Data Engineer: Tech stack - Python, SQL, ETL Pipelines, Airflow, BigQuery, Redshift, Snowflake, AWS/GCP Experience with event-stream and micro-batching data Experienced in Data Quality Assurance Experience with Data Warehouse Modelling Hybrid working in Central London - Monday in the office is essential Pays £450-£500 p/d Outside IR35 3-month initial engagement Senior Data Engineer - Python, SQL, Airflow, Redshift, Snowflake
Responsibilities
- Develop and manage ETL pipelines using Python and SQL.
- Ensure data quality and integrity through rigorous testing and validation processes.
- Collaborate with teams to design and implement data warehouse solutions.
- Utilize Airflow for scheduling and managing workflows.
- Work with cloud technologies such as AWS and GCP.
- Monitor and optimize data processes for performance and reliability.
- Implement best practices for data quality assurance.
Requirements
- Proven experience as a Data Engineer with proficiency in Python and SQL.
- Strong knowledge of ETL processes and data pipeline development.
- Experience with cloud platforms, particularly AWS and GCP.
- Familiarity with data warehousing solutions like Redshift and Snowflake.
- Knowledge of event-streaming and micro-batching data methodologies.
- Understanding of data quality assurance practices.
- Experience with data warehouse modeling techniques.