Overview
We are seeking a skilled Data Engineer to build and maintain scalable data pipelines and support large-scale analytics systems. In this role, you will engage with high-volume data using modern orchestration frameworks and big data technologies, collaborating with cross-functional teams to ensure data quality and reliability.
Responsibilities
- Build and maintain data pipelines using Python.
- Develop and optimize ETL workflows.
- Write complex SQL queries using Presto or Spark SQL.
- Utilize Spark for large-scale data processing.
- Ensure data quality, monitoring, and reliability.
- Collaborate with cross-functional teams including Data Science, Product, and Analytics.
Requirements
- Strong expertise in Python and SQL.
- Experience with data pipeline and orchestration tools such as Airflow.
- Hands-on experience with Spark and big data processing.
- Solid understanding of data warehousing concepts.
- Experience working with large datasets.
- Familiarity with FAANG-scale environments.
- Knowledge of data lineage and metrics frameworks.
- Exposure to real-time monitoring tools.