Overview
Senior Data Engineer - Python, SQL, Airflow, Redshift, Snowflake I've partnered with a high-growth start-up looking to expand their Data function, by bringing on a Senior Data Engineer on a contract basis - more info below: Senior Data Engineer: Tech stack - Python, SQL, ETL Pipelines, Airflow, BigQuery, Redshift, Snowflake, AWS/GCP Experience with event-stream and micro-batching data Experienced in Data Quality Assurance Experience with Data Warehouse Modelling Hybrid working in Central London - Monday in the office is essential Pays £450-£500 p/d Outside IR35 3-month initial engagement Senior Data Engineer - Python, SQL, Airflow, Redshift, Snowflake
Responsibilities
- Develop and maintain ETL pipelines using Python and SQL.
- Collaborate with data analysts to ensure data quality and integrity.
- Implement data warehouse modeling techniques in Redshift and Snowflake.
- Utilize Airflow for workflow orchestration and scheduling.
- Leverage AWS and GCP services for data storage and processing.
- Work with event-stream and micro-batching data processes.
- Monitor and optimize data processing systems for performance.
Requirements
- Proven experience as a Data Engineer with expertise in Python and SQL.
- Strong knowledge of ETL processes and tools, including Airflow.
- Experience with data warehousing solutions such as Redshift and Snowflake.
- Familiarity with BigQuery and cloud services (AWS/GCP).
- Understanding of data quality assurance practices.
- Experience in modeling and optimizing data warehouses.
- Knowledge of event-stream data processing techniques is a plus.