We are recruiting for an experienced Data Engineering Lead, working for one of the UK’s leading technology and cloud services providers, as they continue to invest in a modern, scalable Microsoft Fabric data platform.
This role owns the end‑to‑end data engineering architecture — from ingestion and lakehouse design to enabling trusted, self‑service analytics across the organisation. You will lead a small engineering team, define technical standards, and work closely with senior stakeholders to ensure data underpins strategic and operational decision‑making.
Role Purpose
To take ownership of the organisation’s data platform architecture and delivery, ensuring reliable, scalable ingestion pipelines, a well‑designed data warehouse, and high‑quality analytics products. The role combines hands‑on technical leadership with people management and cross‑business collaboration.
Key Responsibilities
- Own the data warehouse schema, lakehouse design, and core data models
- Translate business requirements into pragmatic, scalable data architecture decisions
- Define, implement, and maintain data contracts aligned to business domains
- Establish data quality, observability, and SLA frameworks
- Design and evolve scalable data pipelines (batch and incremental watermark‑based ingestion) using:
- Microsoft Fabric Pipelines
- PySpark / Python Fabric Notebooks
- Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses
- Ensure Direct Lake–ready semantic models for analytics
- Lead incident management, root cause analysis, and data issue post‑mortems
Analytics Enablement
- Enable self‑service analytics through governed Power BI datasets
- Support and guide the production of trusted Power BI reporting using data contracts
Leadership & Ways of Working
- Define engineering standards and review pull requests
- Mentor and develop 2–4 data and cloud engineers
- Drive the data and warehouse roadmap alongside the Digital Transformation Director
- Champion CI/CD, secure secret management, and software engineering best practices
Role Objective
- Deliver and continuously refine a Microsoft Fabric data warehouse supporting business‑led decision making
- Integrate data from diverse sources including APIs, files, databases, and cloud platforms
- Establish data contracts as a core mechanism for trusted analytics
- Scale Power BI reporting using standardised, high‑quality data products
- Provide clear, motivating leadership to direct reports
- Conduct regular 1:1s, performance reviews, and development planning
- Recruit, onboard, and develop a high‑performing data engineering team
- Ensure compliance with company policies, legal obligations, and operating procedures
Experience & Technical Skills
- Experience leading or mentoring data engineers
- Hands‑on experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models
- Strong Python and PySpark development capability
- Solid SQL skills for warehouse development and metadata layers
- CI/CD pipelines within Azure and Fabric
- Azure Key Vault or similar secret management
- Experience in telecoms or cloud‑focused environments
- Strong understanding of dimensional modelling and medallion architecture
- Experience integrating REST APIs, SFTP, and cloud‑based data sources
- Ability to communicate technical trade‑offs to non‑technical stakeholders
- Excellent troubleshooting, prioritisation, and delivery focus
#J-18808-Ljbffr