Data Engineer (Spark/Scala) - London

Data Engineer (Spark/Scala) - London

Reference Code: 452124-en_GBContract Type:PermanentProfessional Communities: Data & AI

Location:

Northampton (Hybrid 2-3 Days / Week)

About the Job you are considering:

This role involves designing, building, and supporting largescale Big Data solutions using Hadoop and Spark technologies. You will primarily develop and debug Spark jobs in Scala, with opportunities to use Java and Python where appropriate. The position focuses on creating scalable data pipelines, optimizing distributed processing, and supporting analytics and data science teams. It also offers exposure to cloudbased Big Data platforms and modern data engineering practices in a fastpaced, datadriven environment.

Hybrid working:

The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

Your Role:

• Design and develop Hadoopbased applications and scalable data pipelines.
• Debug, develop, and optimize Spark jobs primarily using Scala.
• Build, operate, monitor, and troubleshoot Hadoop clusters.
• Develop robust ETL workflows using Spark, Hive, and Pig.
• Create and manage data ingestion pipelines using Sqoop, Flume, or Kafka.
• Optimize MapReduce jobs and efficiently manage HDFS storage.
• Collaborate with data scientists and analysts to meet data requirements.
• Ensure data security, compliance, and maintain technical documentation.

Your Skills:

• Strong experience developing and debugging Spark applications in Scala.
• Handson Java Big Data development experience.
• Good knowledge of Python for data processing and scripting (nice to have).
• Solid understanding of Hadoop ecosystem: HDFS, MapReduce, YARN.
• Experience with Hive, Pig, and HBase.
• Knowledge of data ingestion tools such as Sqoop, Flume, and Kafka.
• Ability to optimize distributed data processing jobs.
• Exposure to cloud platforms such as AWS, GCP, or Azure is a plus.

We are a Disability Confident Employer:

Capgemini is proud to be aDisability Confident Employer (Level 2)under the UK Governments Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:

• Declare they have a disability, and
• Meet the minimum essential criteria for the role.

Please opt in during the application process.

Make It Real (what does it mean for you):

• Youd be joining an accredited Great Place to work for Wellbeing in 2024. Employee wellbeing is vitally important to us as an organisation. We see a healthy and happy workforce a critical component for us to achieve our organisational ambitions.
  To help support wellbeing we have trained Mental Health Champions across each of our business areas, and we have invested in wellbeing apps such as Thrive and Peppy.
• You will be empowered to explore, innovate, and progress. You will benefit from Capgeminis learning for life mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard ManageMentor, Cybersecurity qualifications and much more.
• You will be joining one of the Worlds Most Ethical Companies, as recognised by Ethisphere for 13 consecutive years. We live our values by making ethical business choices every day. Working ethically is at the centre of our culture at Capgemini, meaning you will be helping to create a future we can all be proud of.

Why you should consider Capgemini:

Growing clients businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, youll join a thriving company and become part of a collective of free-thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine whats possible. Its why, together, we seek out opportunities that will transform the worlds leading businesses, and its how youll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, youll build the skills you want. Youll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is.

About Capgemini:

Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organisations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of 22.1 billion.
Make it real | www.capgemini.com