Responsibilities
- Architect and lead the implementation of complex data automation solutions that replace manual workflows with scalable, reliable data pipelines across the organization
- Design and build end‑to‑end Bronze → Silver → Gold transformation pipelines, implementing domain logic, data quality frameworks, and reproducible data products
- Own the design of the data platform architecture, including ingestion frameworks, transformation patterns, orchestration strategies, and data governance standards
- Establish and enforce data engineering best practices, including schema evolution, data contracts, validation frameworks, and performance optimization across all data pipelines
- Build scalable backend data services and APIs that expose curated datasets to analytics tools, product applications, and downstream consumers
- Lead the integration of data sources from operational systems, external applications, and sensor data, implementing both batch and streaming patterns as needed
- Partner closely with product teams, data analysts, business stakeholders, and AI Engineers to understand complex data requirements and translate them into robust technical solutions
- Drive technical excellence by establishing architectural standards, conducting design reviews, and documenting patterns that enable team scalability and maintainability
- Evaluate and recommend data technologies, tools, and platforms that will improve engineering productivity and data platform capabilities
Qualifications
- A degree in Computer Science, Mathematics, or a related field
- 5+ years of experience as a Cloud Engineer, Data Engineer, Backend Engineer, or similar role with demonstrated technical leadership
- Proven experience architecting and maintaining complex data pipelines, transformation workflows, and data platform components that serve multiple downstream consumers
- Proven experience building event‑driven and streaming data integrations at scale
- Expert knowledge of cloud data services across either Azure or AWS ecosystems (e.g., ADLS, ADF, Databricks, Event Hubs, S3, Glue, EMR, Kinesis, Lambda)
- Production experience deploying and managing containerized services in Kubernetes environments (AKS and/or EKS preferred)
- Strong expertise with SQL and NoSQL systems (e.g., Postgres, MongoDB, Redis) including schema design and optimization
- Knowledge of data governance processes and tools and metadata management solutions
- Experience with infrastructure as code and platform engineering practices (e.g., Terraform)
- Experience implementing and maintaining CI/CD workflows (Azure Pipelines, GitHub Actions, AWS CodePipeline/CodeBuild, ArgoCD)
- Strong understanding of access control frameworks and security governance in hybrid environments, including integration with enterprise identity providers (e.g., Active Directory/LDAP, SSO, IAM federation)
- Good programming skills in Python
- Excellent communication and leadership skills, with proven ability to influence technical direction
Preferred Qualifications
- Strong experience designing and operating data platforms supporting telemetry, sensor, location, and operational technology (OT) data workloads
- Experience processing and managing large‑scale video, image, and unstructured media datasets, including streaming ingestion and analytics workflows
- Understanding of feature engineering patterns and data preparation workflows supporting machine learning systems
- Strong understanding of private cloud and on‑premises environments, including secure service design, network isolation, and hybrid connectivity patterns between on‑prem and public cloud
- Experience working with maritime, fleet, vessel operations, logistics, or industrial operational domains, including telematics, tracking, asset monitoring, or operational analytics use cases
Salary
Up to £95,000 per annum
#J-18808-Ljbffr…
