Data Engineering Lead

Data Engineering Lead

Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract to the end of the year and is required to be onsite 3 days a week.

The successful candidate will lead the data acquisition function by defining ingestion strategy and building Azure-based data pipelines. You will translate end‑user requirements into scalable engineering solutions, act as the domain’s primary Data Manager for data governance, lineage, quality, and master data management, and lead a team of data engineers in delivering reliable, compliant, and high‑value data solutions to Trading, Risk, Finance, and Operations.

Responsibilities

• Own ingestion strategy and define scalable batch/streaming data patterns
• Design and deliver Azure-based data pipelines and platform solutions
• Standardize connectors, schemas, controls, CI/CD, IaC, and automated testing
• Lead onboarding of enterprise and market data sources with SLAs and observability
• Partner with Product Owners, Architects, and business stakeholders on data design and governance
• Act as domain Data Manager for lineage, metadata, quality, access, and master data ownership
• Ensure compliance with data lifecycle, audit, records management, and governance standards
• Drive engineering best practices, operational excellence, and continuous improvement
• Manage capacity planning, skills development, and vendor contributions

Qualifications

• Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark
• Experience with data modeling, metadata and lineage tooling; performance & cost optimization
• Proven leadership in engineering teams; Agile/DevOps ways of working; CI/CD; test automation
• Exposure to energy trading systems (ETRM/Endur), market data (ICE, Platts, Bloomberg), SAP
• Degree in Computer Science/Engineering or equivalent experience; Azure Data Engineer or Databricks certifications a plus

#J-18808-Ljbffr…