Overview
As a Data Engineer, you will be the first data hire in a rapidly growing professional services firm focused on AI-enabled solutions. Working closely with the engineering lead, you will build and enhance a Databricks lakehouse from the ground up, establishing standards for data ingestion and governance as the platform evolves. Your work will directly contribute to the organization’s data strategy and operational excellence.
Responsibilities
- Build ingestion pipelines into the Databricks lakehouse.
- Model data through medallion layers in Delta Lake.
- Develop reconciliation and data quality checks within pipelines.
- Set up data governance using Unity Catalog.
- Harden data pipelines for production use.
Requirements
- Proven hands-on experience with Databricks, including Delta Lake, PySpark, and Spark SQL.
- Strong proficiency in Python with a focus on production environments in a major cloud platform.
- Ability to manage workstreams end-to-end with minimal oversight.
- Effective communication skills with a commercial awareness.
- Familiarity with Databricks AI/ML stack (MLflow, Vector Search, Mosaic AI) is a plus.
- Experience with RAG or LLM-powered data extraction is advantageous.