Overview
The Principal AWS Data Engineer will support an AI/ML startup in a go-live project over a 6-week contract, with potential for extension. This fully remote role involves hands-on engineering work as well as reviewing architectural designs, focusing on creating and managing real-time data pipelines at scale. The contractor will be expected to collaborate closely with a dynamic team in a fast-paced environment, bringing in substantial AWS expertise and a strong architectural background.
Responsibilities
- Design and implement real-time data pipelines at scale.
- Review and enhance data architecture to improve efficiency.
- Develop solutions using Amazon Web Services (EKS, S3, RedShift).
- Collaborate with teams to integrate Kafka Flink or equivalent technologies.
- Conduct testing of data pipelines to ensure reliability and performance.
- Utilize Python or Java for data engineering tasks.
- Participate in CI/CD processes for data projects.
Requirements
- Proven experience in AWS services, particularly EKS, S3, and RedShift.
- Strong understanding of data architecture principles.
- Experience with real-time data processing technologies like Kafka Flink or equivalent.
- Proficiency in Python or Java programming languages.
- Familiarity with AI/ML concepts and applications.
- Experience in testing and maintaining data pipelines at scale.
- Ability to thrive in fast-paced startup environments.