Overview
We are looking for an experienced AWS Data Engineer contractor who can contribute to a significant cloud transformation initiative focused on migrating legacy ETL processes to a modern AWS data platform. This role involves hands-on participation in defining migration patterns and implementing production-ready solutions, while working closely with a team dedicated to this financial services project.
Responsibilities
- Lead the migration of batch jobs from Ab Initio to AWS Glue solutions.
- Define and implement AWS Glue migration patterns and processes.
- Develop and optimize large-scale ETL/data pipeline solutions in AWS.
- Collaborate with team members to troubleshoot performance issues related to Spark and data transformations.
- Utilize advanced Python, PySpark, and SQL to build scalable data processing solutions.
- Create data warehousing and modeling strategies as needed.
- Participate in a remote-first working environment with occasional on-site collaboration.
Requirements
- Proven experience as an AWS Data Engineer with strong hands-on expertise.
- Experience in migrating ETL processes from Ab Initio to AWS Glue (PySpark).
- Strong proficiency in AWS technologies, particularly Glue, S3, and Athena.
- Advanced skills in Python, PySpark, and SQL for data engineering tasks.
- Comprehensive understanding of Spark optimization techniques and data transformations.
- Excellent knowledge of data warehousing concepts and data modeling principles.
- Strong communication skills to collaborate effectively in a distributed team environment.
- AWS Certification is preferred but not mandatory.