Job Overview
This role is key to supporting the design, build, and validation of data migration pipelines, enabling the successful transition of data from legacy systems to modern cloud platforms. You will contribute to ensuring data quality, integrity, and performance, particularly through structured testing and validation activities. You will work closely with architects, senior engineers, and analysts to deliver scalable and reliable migration solutions, using technologies such as AWS Glue, Apache Iceberg, Python/PySpark, SQL, and YAML configurations. You should be comfortable working in a collaborative, delivery-focused environment and have a strong interest in data migration, cloud technologies, and modern data engineering practices.
Client Engagement & Delivery
- Support delivery within data migration programmes, contributing to key workstreams.
- Collaborate with architects, engineers, and stakeholders to implement migration solutions.
- Assist in planning and executing data migration tasks and deliverables.
Data Migration Engineering
- Build and maintain data migration pipelines from legacy data warehouses to AWS-based platforms.
- Develop ETL/ELT pipelines using AWS Glue, Python/PySpark, SQL, and YAML configurations.
- Support execution of bulk data migrations and incremental/delta loads.
- Assist with pipeline repointing and migration to cloud environments.
Data Pipeline Testing & Validation (Core Focus)
- Test ETL/ELT data pipelines on AWS services, including AWS Glue and Apache Iceberg.
- Support validation of data pipeline migrations to AWS Data Lakehouse architectures.
- Execute and validate initial bulk data loads and incremental/delta data processing.
- Write and run SQL queries to validate data completeness, data accuracy, and transformation outputs.
- Support development of test scripts and validation checks.
AWS Data Platforms & Lakehouse
- Work with AWS services including AWS Glue and S3-based data lakes.
- Support implementation of Data Lakehouse architectures, including Apache Iceberg.
- Contribute to improving pipeline performance and reliability.
Data Transformation & Support
- Apply transformation logic based on defined data mapping rules.
- Support preparation of data for target-state models.
- Assist in ensuring consistency between source and target datasets.
Collaboration & Best Practices
- Work collaboratively with solution architects, data engineers, data migration architects, analysts, and QA teams.
- Follow established engineering standards and best practices.
- Contribute to documentation and reusable components.
Quality, Governance & Security
- Support maintenance of data quality and integrity during migration.
- Follow secure data handling practices.
- Assist with compliance requirements, including GDPR and public sector data standards where applicable.
- Contribute to testing, validation, and audit activities.
Qualifications
- Experience in data engineering or data migration delivery.
- Strong focus on testing, validation, and data quality assurance.
- Ability to work across data pipelines and transformation workflows.
- Good analytical and problem-solving skills.
- Effective communication and teamwork skills.
- Willingness to learn and develop in data migration and cloud technologies.
- Hands-on experience with AWS cloud services, especially AWS Glue, Python/PySpark, SQL querying and validation, YAML configuration.
- Experience testing or supporting ETL/ELT pipelines and data migration processes.
- Familiarity with data lake/lakehouse concepts (e.g., Apache Iceberg) and distributed processing frameworks (e.g., Spark).
- Basic understanding of ETL vs ELT approaches and cloud-based data architectures.
- Exposure to version control and CI/CD tools desirable.
Benefits
We offer a range of tailored benefits that support your physical, emotional, and financial wellbeing. Our Learning and Development team ensures continuous growth and development opportunities. Flexible work options are also available. Inclusion networks such as Women’s Business Network, Cultural and Ethnicity Network, LGBTQ+ & Allies Network, Neurodiversity Network and Parent Network provide support and community.
Equal Opportunity Employer
We are an equal opportunities employer. We believe in the fair treatment of all our employees and commit to promoting equity and diversity in our employment practices. We are also a proud Disability Confident Committed Employer – we are committed to creating a diverse and inclusive workforce. We actively collaborate with individuals who have disabilities and long-term health conditions which have an effect on their ability to do normal daily activities, ensuring that barriers are eliminated when it comes to employment opportunities. In line with our commitment, we guarantee an interview to applicants who declare to us, during the application process, that they have a disability and meet the minimum requirements for the role. If you require any reasonable adjustments during the recruitment process, please let us know.
#J-18808-Ljbffr…