Staff Data Engineer

Salary: £? – ? per year

Requirements

• We are looking for experience in Staff, Principal or Lead Data Engineering roles.
• We need strong Python skills and expert-level PySpark experience.
• We require experience with AWS Glue, Apache Spark and Apache Airflow.
• We are looking for experience with cloud‑native Data Engineering architectures.
• We need experience delivering large‑scale data pipeline modernisation.
• We value experience building reusable engineering patterns within enterprise environments.
• We require experience in data testing, validation and observability.
• We are looking for experience using AI‑assisted software development tools and modern coding assistants.
• Experience working within regulated or highly governed organisations is important.

Responsibilities

• We are defining technical standards and engineering patterns for a large‑scale Data Modernisation programme.
• We are re‑engineering complex legacy data pipelines into production‑grade PySpark.
• We are building scalable, reusable data movement frameworks across multiple engineering teams.
• We are driving best practice around AWS Glue, Apache Spark and Apache Airflow.
• We are designing high‑quality testing frameworks covering performance, data quality and reliability.
• We are leveraging Agentic AI development tools to improve engineering productivity while maintaining code quality.
• We are coaching and influencing experienced engineers through technical leadership rather than line management.
• We are working closely with enterprise architecture and platform teams to improve how data is governed, managed and consumed.

Technologies

• AI
• Airflow
• AWS
• AWS Glue
• Cloud
• Python
• PySpark
• Spark

More

We are supporting a global, technology‑led organisation undertaking a large‑scale cloud data transformation programme. This is a Staff‑level contract opportunity for an engineer who can combine deep hands‑on technical expertise with organisation‑wide technical influence across multiple engineering teams. The role focuses on enterprise‑scale data movement, cloud‑native architectures and AI‑assisted software engineering within a highly collaborative engineering culture. The contract is initial 6 months, inside IR35, based in London with 3 days per week on‑site.

last updated 29 week of 2026

#J-18808-Ljbffr…