Data Engineering Lead (Data Acquisition)

Overview

What’s the role

As a Data Engineering Lead (Data Acquisition), you will lead Shell Energy Trading & Supply’s data acquisition function: own the ingestion strategy and build pipelines on Azure; act as the bridge from end-user needs to engineered solutions; serve as the domain’s go-to Data Manager for mastership, lineage, and quality; and lead a team of data engineers to deliver reliable, compliant, high-value data to Trading, Risk, Finance, and Operations.

Responsibilities

• Own the ingestion strategy and build high-quality pipelines on Azure (ADLS, ADF, Databricks, Synapse, Event Hubs/Kafka) with CI/CD, IaC and automated testing.
• Define target-state patterns for batch/streaming ingestion; standardize connectors, schemas, contracts and controls.
• Industrialize onboarding of sources (ETRM/Endur, Trayport, market data vendors, SAP, SCADA/IoT) with SLAs, observability and runbooks.
• Translate trading desk, risk, operations and finance requirements into a prioritized acquisition backlog; shape epics/stories with clear acceptance criteria.
• Partner with Product Owners, Business Interface and Solution/Data Architects to align designs, models and mastership decisions for the domain.
• Steward data definitions, ownership, lineage, metadata, quality KPIs and access policies; chair data change forums.
• Apply Shell’s technical information & data lifecycle: register, receive, set up/load, update/maintain, publish/archive or dispose; uphold audit trail and records management.
• Lead and develop a team of Data Engineers; build a high-performing, inclusive team; coach in coding standards, design reviews, performance tuning and production operations.
• Shape capacity plans, skills roadmap and vendor contributions; foster a culture of safety, compliance and continuous improvement.
• Operate reliably & safely: ensure resilience, recoverability, cyber hygiene; meet controls for data privacy, confidentiality and regulatory obligations in trading (e.g., REMIT/EMIR).
• Drive incident, change & problem management; measure and report service health and user satisfaction.

Qualifications

• Extensive experience in data engineering, with proven years leading teams/products in complex, real-time data environments.
• Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark.
• Strong knowledge of streaming and batch patterns, schema management, CDC, API integration.
• Experience with data modelling, metadata and lineage tooling; performance & cost optimization.
• Proven leadership in engineering teams; Agile/DevOps ways of working; CI/CD; test automation.
• Preferred: exposure to energy trading systems (ETRM/Endur), market data (ICE, Platts, Bloomberg), SAP.
• Degree in Computer Science/Engineering or equivalent experience; Azure Data Engineer or Databricks certifications a plus.

Note

We are committed to attracting a broader and more diverse pool of candidates. If this position doesn’t feel like the perfect fit for your qualifications right now, we’d still love to hear from you. Shell is an Equal Opportunity Employer.

#J-18808-Ljbffr…