Data Engineer (Azure)

Data Engineer FTC – 12 Months – Coventry

University of Warwick

• £47,389 - £56,535 (D.O.E)
• 42 Days Holidays (Inc Bank Hols)
• Excellent Pension Contributions
• Hybrid Working
• Great Food, Gym and Leisure Facilities on-site

We are seeking a new addition for our Data Labs Team to assist in the continued development and transformation processes that will ensure Warwick remains a leading and world recognised university for academia and research.

As the Data Engineer, you will be responsible for leveraging Azure Data Factory and DBT to design, develop, and maintain robust data pipelines and scalable data models. Your work will directly contribute to enhancing data accessibility, quality, and analytics capabilities within the university.

Duties and Responsibilities

Technical work to model data and Design and Develop Data Pipelines:

▪ Utilize Azure Data Factory to design and develop scalable and efficient data pipelines, encompassing data extraction and loading processes. Integrate data from various sources, including internal systems, third-party applications, and external data providers.

▪ Utilize DBT (Data Build Tool) to create and manage data transformation processes, ensuring consistent and reliable data output.

▪ Implement and maintain DataVault and Kimball-style data models to ensure efficient storage, retrieval, and analysis of data.

▪ Monitor and optimize the performance of data pipelines and data warehouses, identifying and resolving bottlenecks and inefficiencies.

▪ Implement data security measures, including access controls and encryption, to safeguard sensitive information. Ensure compliance with relevant data privacy and protection regulations, such as GDPR.

Data Quality Assurance

▪ Implement data quality checks and validation processes within the pipelines to ensure accuracy, completeness, and consistency of data. Collaborate with data governance, architects, system owners and end users to ensure quality issues are addressed as needed.

Documentation

▪ Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic, to facilitate knowledge sharing and support future enhancements.

Collaboration

▪ Collaborate with cross-functional teams, including data analysts, data scientists, and business stakeholders, to understand requirements and deliver solutions that meet their data needs.

Requirements

• Degree in Computer Science, Data Engineering, or a related field or equivalent experience.
• Strong experience working as a Data Engineer, preferably in a cloud-based environment.
• Experience with cloud-based data storage platforms, preferably Snowflake, Azure SQL Data Warehouse or AWS Redshift.
• Solid practical experience and understanding of DataVault and Kimball-style data warehousing methodologies.
• Proficient in SQL and data querying languages for data manipulation and analysis.
• Proficiency in Azure Data Factory and DBT, with a demonstrated ability to build scalable and reliable data pipelines and transformation processes.
• Familiarity with data modelling concepts and techniques, including dimensional modelling.
• Strong problem-solving and analytical skills, with the ability to analyse complex data-related issues and provide effective solutions.
• Excellent communication and collaboration skills, with the ability to work effectively in a team environment and engage with stakeholders at various levels.