Overview
The Forward Deployed Engineer will play a crucial role in a long-term enterprise data transformation programme, working hands-on with data engineering, architecture, and advanced data modeling. This role involves close collaboration with business stakeholders to translate complex requirements into technical solutions, focusing on optimizing data workflows and infrastructure within Databricks.
Responsibilities
- Design, optimise, and operate Apache Spark workloads across batch and streaming performance tuning.
- Architect and deliver Delta Live Tables pipelines and Databricks governance models.
- Build and maintain infrastructure-as-code using Terraform and CI/CD automation via GitHub Actions.
- Design data models and implement query optimisation and row-level security strategies.
- Develop full-stack application components for the NEXUS platform including deployment via Databricks Apps.
- Interoperate between Databricks and on-prem/custom data platforms.
- Take ownership of workstreams and interact directly with stakeholders to define solutions.
Requirements
- Strong experience in enterprise data modeling and architecture.
- Advanced SQL and strong Python/PySpark skills.
- Hands-on experience with Databricks, including Delta Lake and Workflows.
- Production experience with dbt and Spark Declarative Pipelines.
- Understanding of enterprise data architecture, handling complex data challenges.
- Ability to translate business requirements into technical solutions.
- Experience with data governance, access controls, and security.
- Familiarity with Terraform and CI/CD practices.