Senior Data Engineer

Overview

Lead Data Engineer (Databricks Specialist) Contract: 3–6 months Day Rate: £600–£700 (Outside IR35) Start Date: January 2026 Location: UK (Remote / Hybrid options) We are representing a Databricks delivery partner who help organisations transform the way they use data, building scalable, modern data platforms and accelerating AI adoption through Databricks. We’re looking for a Lead Data Engineer who is exceptional with Databricks and ready to play a key role in designing, building, and delivering cutting-edge data solutions for enterprise clients. This is a hands-on role for a seasoned engineer who can operate across architecture, delivery, and consulting. Key responsibilities include: Design and build end-to-end data solutions on Databricks, using Spark, Python, SQL, and Delta Lake Apply software engineering best practices — TDD, CI/CD, version control, automation, and clean coding principles Work across the entire software development lifecycle, from design to deployment Engage directly with stakeholders, shaping requirements and influencing technical direction Lead delivery of complex data engineering initiatives, individually or with small teams Provide technical leadership, mentoring others, reviewing code, and driving best practices Collaborate with data scientists, architects, and business teams to deliver production-grade outcomes Essential skills needed: Deep hands-on experience with Databricks (SQL, PySpark, Delta Lake, Unity Catalog, Workflows) Strong proficiency in Python and Spark Solid understanding of CI/CD pipelines, DevOps, and Infrastructure as Code Proven track record designing and delivering solutions from scratch Excellent communication and consulting skills, comfortable working with both technical and business stakeholders Strong problem-solving ability, commercial awareness, and a proactive approach to delivery Start Date: January 2026 Duration: 3–6 months Rate: £600–£700 per day (Outside IR35) Location: UK-based (remote or hybrid depending on client) If you’re a Databricks expert who thrives on solving complex problems, delivering real impact, and setting engineering standards, click apply now.

Responsibilities

• Design and build end-to-end data solutions on Databricks using various technologies.
• Apply software engineering best practices, including TDD and CI/CD.
• Engage with stakeholders to shape requirements and technical direction.
• Lead delivery of complex data engineering projects individually or with teams.
• Provide technical leadership and mentorship to other engineers.
• Collaborate with cross-functional teams to achieve production-grade outcomes.
• Review code and drive best practices in data engineering.

Requirements

• Deep hands-on experience with Databricks, including SQL, PySpark, and Delta Lake.
• Strong proficiency in Python and Spark.
• Solid understanding of CI/CD pipelines and DevOps practices.
• Proven track record in designing and delivering solutions from scratch.
• Excellent communication skills for interacting with technical and business stakeholders.
• Strong problem-solving ability with a proactive delivery approach.
• Experience with Infrastructure as Code and automation tools is a plus.