Overview
The Data Quality Analyst will play a critical role in a technical project focused on testing and ensuring the integrity of complex, high-volume data pipelines. The contractor will leverage their expertise in ETL/ELT processes and data validation to build and execute testing frameworks, while collaborating with technical teams to validate data flows and integrations. The position is remote and expected to last for at least six months, starting as soon as possible.
Responsibilities
- Conduct hands-on testing of high-volume data pipelines and ETL/ELT processes.
- Build and execute data validation frameworks to ensure data accuracy.
- Perform source-to-target reconciliation to verify data integrity.
- Utilize tools such as Kafka and Apache Spark for data validation tasks.
- Develop mocks and test doubles for integration and pipeline testing.
- Validate APIs and data flows using tools like Postman.
- Document test cases, results, and evidence of testing activities.
Requirements
- Proven experience in testing high-volume, production-scale data pipelines.
- Strong hands-on knowledge of ETL/ELT processes and reconciliation techniques.
- Proficiency in data validation using SQL and/or Python.
- Experience with Kafka and streaming data technologies.
- Familiarity with Kubernetes and containerized environments.
- Solid background in API testing with tools such as Postman.
- Strong documentation skills related to test development and execution.