The Opportunity
Join a team building the data foundations that support the firm’s AI and analytics capabilities. This role sits within the engineering effort to develop a modern Lakehouse and AI data platform that enables reliable, well‑governed and high‑performing data use across the firm.
In this delivery‑focused engineering role you will design, build, test and support data pipelines and curated datasets on the firm’s modern data platform. You will work across ingestion, transformation, modelling, optimisation and data quality to deliver reliable, scalable, and fit‑for‑purpose data products. This position is suited to engineers comfortable writing code, using SQL, distributed processing, and tackling practical delivery problems in a team.
Role Summary
As a Data Engineer, Lakehouse and AI Data Platform, you will contribute to the data models, pipelines and platform capabilities that underpin analytics, operational decision‑making and emerging AI use cases.
Key Responsibilities
- Pipeline Engineering
- Build, enhance and support batch and streaming data pipelines on the Lakehouse and AI data platform.
- Refactor or modernise existing data flows to improve reliability, performance and maintainability.
- Ensure data pipelines are production‑ready, well‑tested and operationally supportable.
- Data Modelling and Curation
- Develop raw, refined and curated datasets that support analytics, reporting and AI use cases.
- Apply sound data modelling principles to represent business entities, relationships and historical change accurately.
- Work with consumers to shape data products that are usable, well documented and aligned to business needs.
- Data Quality and Reconciliation
- Implement controls to validate completeness, accuracy and consistency of data across pipelines and datasets.
- Use reconciliation approaches to build confidence in production outputs and investigate breaks where they arise.
- Contribute to clear standards for testing, monitoring and issue resolution.
- Delivery and Partnership
- Work closely with engineers, platform teams and data consumers to deliver agreed outcomes on time and to quality expectations.
- Communicate clearly on progress, risks, dependencies and design choices.
- Take a broader role in technical leadership, task breakdown and support for junior engineers.
Required Skills and Experience
- Bachelor’s or master’s degree in a relevant discipline, or equivalent practical experience, with evidence of strong quantitative skills or data engineering expertise.
- Strong hands‑on programming experience in Python or Java.
- Good working knowledge of SQL, including troubleshooting, optimisation and data analysis.
- Ability to learn new tools, internal platforms and delivery workflows quickly.
- Familiarity with software engineering fundamentals, including version control, testing, release discipline and CI/CD practices.
Data Engineering Capability
- Stronger ownership of technical design across multiple datasets or pipeline domains.
- Experience guiding implementation standards, code quality and engineering practices within a team.
- Ability to lead delivery for a workstream, manage dependencies and support less experienced engineers.
- Understanding of temporal data modelling, including the handling of historical state and change over time.
- Knowledge of schema design, schema evolution and data compatibility considerations.
- Understanding of partitioning, clustering and other techniques used to improve data performance at scale.
- Ability to make sensible design choices across normalised and denormalised models, and between natural and surrogate keys.
- Practical approach to data quality, reconciliation and root‑cause analysis.
- Experience building or supporting production data pipelines in a collaborative engineering environment.
- Experience working with distributed data processing frameworks such as Apache Spark.
- Working knowledge of common data formats such as JSON, Avro and Parquet.
Technology Environment
The role will involve working with a modern and evolving data stack. Candidates are not expected to have deep expertise in every tool from day one but should bring relevant experience and the ability to work across comparable technologies.
Examples of Technologies In Scope
- Data processing and logic: ANSI SQL, Apache Spark, Kafka
- Data formats: JSON, Avro, Parquet
- Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ
- Engineering and deployment: CI/CD tooling, containerised or Kubernetes‑based deployment approaches where relevant.
What We Are Looking For
We are looking for engineers who can deliver well‑structured, reliable solutions in production and who take ownership of the quality of what they build. The role suits candidates who are technically strong, pragmatic and comfortable working in a fast‑paced environment where data platforms support important business outcomes.
Stronger Candidates Will Typically Demonstrate
- Sound judgement in technical trade‑offs
- Attention to detail in data correctness and testing
- A clear and structured approach to problem solving
- Willingness to work closely with stakeholders and partner teams
- An interest in developing long‑term expertise within the firm.
Goldman Sachs is an equal opportunity employer and does not discriminate on the basis of race, color, religion, sex, national origin, age, veterans status, disability, or any other characteristic protected by applicable law.
#J-18808-Ljbffr