Overview
The role of a Data Platform Engineer involves developing and maintaining a comprehensive data observability platform that ensures transparency and control over data pipelines. The contractor will be primarily focused on enhancing data quality frameworks, optimizing cloud data platform performance, and supporting engineering teams through the development of reusable tools. This position allows for collaboration with advanced data technologies and contributes to the overall efficiency and reliability of data management processes.
Responsibilities
- Build and deploy a comprehensive Data Observability platform to track end-to-end data pipelines.
- Design and implement reusable components and automated testing frameworks to ensure data quality.
- Monitor, tune, and optimize compute and storage costs within Snowflake.
- Develop reusable frameworks and automation tools to support engineering teams.
- Manage and support a modern data tooling ecosystem including Snowflake and integration platforms.
- Assist with administration and optimization of analytics tools such as Power BI and ThoughtSpot.
Requirements
- Proven experience in data engineering with hands-on experience in building scalable data pipelines.
- Strong SQL skills and working knowledge of Python.
- Experience in implementing Data Observability and Data Quality frameworks.
- Solid experience in managing and supporting cloud data platforms.
- Excellent problem-solving abilities and operational focus.
- Exposure to infrastructure as code tools, particularly Terraform.
- Familiarity with Linux environments and modern data tools like Talend, Matillion, and ThoughtSpot is a plus.