Overview
The Senior Data Engineer will play a critical role in architecting and building an AWS Lakehouse from the ground up in a greenfield project environment. Working closely with the Head of Data and Lead Data Engineer, the contractor will influence design decisions, data structure, and scaling, using a modern tech stack that includes AWS components and data engineering frameworks. The role offers a unique opportunity to shape a new data platform rather than maintaining existing systems, with a commitment to a four-day working week.
Responsibilities
- Architect an AWS Lakehouse solution from the ground up.
- Collaborate directly with the Head of Data and Lead Data Engineer.
- Develop and structure data models using AWS tools and modern engineering practices.
- Implement orchestration and governance strategies for data management.
- Engage in key engineering decisions surrounding data lineage and security.
- Utilize and integrate tools such as Python, Terraform, PostgreSQL, and more.
Requirements
- Proven experience as a Data Engineer, particularly in AWS environments.
- Strong skills in AWS services including S3, Glue, Athena, and EKS.
- Familiarity with data orchestration tools such as Dagster or Airflow.
- Experience with data modeling, governance, and security best practices.
- Knowledge of modern data storage formats like Parquet or Delta Lake.
- Understanding of RAG, LangChain, pgvector, FastAPI, and MLflow is a plus.
- Ability to work collaboratively in a fast-paced, team-oriented environment.