Overview
The Senior Data Engineer will play a key role in enhancing the data operations of a high-growth start-up. This contractor will work closely with various teams to develop and maintain robust data pipelines and ensure data quality across multiple platforms, utilizing a diverse tech stack including Python, SQL, and cloud solutions such as AWS and GCP. The position involves a hybrid work model with essential in-office presence on Mondays.
Responsibilities
- Develop and maintain ETL pipelines to process large datasets.
- Ensure data quality and integrity across all projects.
- Collaborate with cross-functional teams to identify data needs and solutions.
- Optimize data workflows using technologies such as Airflow and BigQuery.
- Design and implement data warehouse models for efficient querying.
- Monitor and troubleshoot data processes to improve reliability and efficiency.
- Utilize Python and SQL to perform data analysis and reporting.
- Participate in code reviews and contribute to best practices in data engineering.
Requirements
- Proven experience as a Data Engineer with expertise in Python and SQL.
- Strong knowledge of ETL processes and data pipeline development.
- Experience with event-stream processing and micro-batching techniques.
- Familiarity with cloud platforms like AWS and GCP.
- Hands-on experience with data quality assurance practices.
- Proficiency in data warehouse technologies such as Redshift and Snowflake.
- Ability to work collaboratively in a hybrid team environment.
- Strong analytical and problem-solving skills with attention to detail.