Data Engineer – Databricks and Python
Location: Glasgow, Scotland, United Kingdom
Employment type: Full-time | Seniority level: Associate
We are the world's largest independent renewable energy company, dedicated to providing affordable, zero‑carbon energy. As part of our Digital Solutions business, we are seeking a skilled Data Engineer with expertise in Databricks to build and optimise scalable data pipelines that enable data‑driven analytics and machine learning for our asset performance management software.
This is a 24‑month fixed‑term contract.
Responsibilities
- Design, develop, and maintain robust data pipelines using DLT on Databricks.
- Collaborate with software engineers, data scientists and platform engineers to understand data requirements and deliver high‑quality solutions.
- Implement ETL/ELT processes to ingest, transform, and store data from various sources (structured and unstructured).
- Optimize performance and cost‑efficiency of data workflows on Databricks.
- Ensure data quality, integrity, and governance through validation, monitoring, and documentation.
- Develop reusable components and frameworks to accelerate data engineering efforts.
- Support CI/CD practices and automation for data pipeline deployment.
- Stay current with Databricks features and best practices, and advocate for their adoption.
Knowledge
- Solid understanding of data modelling, warehousing concepts, and distributed computing.
- Familiarity with Delta Lake and Unity Catalog.
- Knowledge of data governance frameworks and compliance standards (e.g., GDPR, HIPAA).
Skills
- Strong programming skills in Python and SQL.
- Experience with version control (e.g., Git) and CI/CD tools.
- Excellent problem‑solving and communication skills, both written and oral.
Experience
- Proven experience as a Data Engineer with hands‑on expertise in Databricks and DLT.
- Experience with cloud data platforms, ideally Azure; experience with AWS or Google is an advantage.
- Exposure to machine learning workflows and integration with ML models.
- Delivering results working in a distributed, cross‑functional team.
Qualifications
- Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).
At RES we celebrate difference and believe that diverse perspectives drive innovation. We encourage applicants with different backgrounds, ideas and points of view to apply. That is why we have a strong commitment to equal opportunity and to creating a welcoming workplace for everyone, regardless of ethnicity, culture, gender, nationality, age, disability, sexual orientation, gender identity, marital or parental status, education, or social background.
We are an equal‑employment‑opportunity employer who strives for inclusion for all employees and applicants.
#J-18808-Ljbffr