Overview
We are seeking a Senior AI Data Engineer to join a dynamic technical consultancy focused on AI adoption within enterprise environments. In this role, you will work closely with our delivery teams to design and implement data pipelines that are vital for AI systems, particularly in managing complex healthcare data. You will leverage your expertise to enhance the capabilities of fellow engineers and contribute to the overall success of data-driven AI transformations.
Responsibilities
- Design and build data pipelines that transform unstructured enterprise data into structured formats suitable for AI consumption.
- Develop chunking and parsing strategies to optimize AI data retrieval in production settings.
- Maintain and improve ingestion pipelines connected to AWS Bedrock Knowledge Bases and retrieval infrastructure.
- Monitor data quality, freshness, coverage, and access controls over time.
- Collaborate with AI engineers and architects to ensure alignment between data layers and AI system requirements.
- Coach and mentor data engineers transitioning into AI data roles, providing guidance on retrieval, chunking, evaluation, and governance practices.
- Document technical processes and decisions to enhance team knowledge and maintain high engineering standards.
Requirements
- Proven experience in data engineering with a solid understanding of AI data consumption methodologies.
- Hands-on experience with AWS data services like S3, Glue, Lambda, and RDS, along with AI services like Bedrock and Knowledge Bases.
- Strong ability to transform unstructured data for AI applications in production.
- Expertise in building end-to-end retrieval pipelines and ensuring their reliability in production environments.
- Familiarity with data governance and compliance practices in enterprise settings.
- Experience working in Kanban delivery environments and managing workflows independently.
- Confidence in using AI development tools effectively in project workflows.