Senior Data Engineer

Overview

We are seeking a Senior Data Engineer for a 12-month Outside IR35 contract role within a leading global commodities trading organization. The role focuses on developing critical data infrastructure to support traders and risk teams, enhancing their capability to access accurate, real-time trading information. You will collaborate with various stakeholders, including traders, risk managers, and senior engineering teams, on this technology programme central to the trading operations.

Responsibilities

  • Design, build and own high-performance Python data pipelines supporting intraday and end-of-day P&L.
  • Ingest and process market data, including settlement prices, broker marks and forward curves.
  • Drive market data quality by identifying missing or incorrect prices before they impact trading and risk calculations.
  • Orchestrate flash and end-of-day P&L runs using a risk engine.
  • Reconcile P&L against trading desk and trader feeds, investigating discrepancies.
  • Build and maintain scalable cloud-based services using AWS and Kubernetes.
  • Contribute to engineering standards across testing, CI/CD, monitoring and automation.

Requirements

  • Extensive commercial Python data engineering experience.
  • Hands-on experience with Apache Airflow or similar workflow orchestration tools.
  • Understanding of commodity or financial market data, including futures, swaps, settlement prices and forward curves.
  • Working knowledge of trading P&L, mark-to-market and P&L.
  • Strong SQL skills and experience with PostgreSQL, ClickHouse or similar databases.
  • Experience supporting business-critical data processes within a front-office or risk environment.
  • Desirable experience with Open Source Risk Engine (ORE), QuantLib or similar valuation engines.
  • Familiarity with market data providers such as LSEG, Argus, Platts or ZEMA, and technologies like AWS, Docker, Kubernetes, Kafka and CI/CD.
OIR35+ members get up to 7 days early access to new jobs
SkillsData Engineering, CI/CD
TechnologiesPython, AWS, Kubernetes, Airflow, SQL, PostgreSQL, Docker, Kafka
LocationLondon Area
TypeOn-site
PostedNEW