Senior Product Manager for AI Observability

Senior Product Manager – AI Observability

As part of the LSEG AI, we are hiring a Senior Product Manager – AI Observability to own the strategy, design and rollout of telemetry systems that monitor, measure and analyse how AI models, MCPs, and AI‑enabled features behave across all LSEG products.

Responsibilities

Telemetry & Observability Strategy

• Define the end‑to‑end telemetry vision and roadmap for LLMs, MCPs, vector stores, embeddings, inference layers and AI‑powered user experiences.
• Establish a standardised telemetry schema for capturing prompts, tool calls, model responses, durations, errors, confidence signals, and quality indicators across business lines.
• Partner with platform engineering to ensure instrumentation is consistent, scalable and compliant.

Signal Design & Data Architecture

• Identify key signals required for quality & reliability, latency & throughput, cost tracking & optimisation, MCP usage patterns, user workflow insights, failure pattern detection, guardrail and safety event monitoring.
• Define retention rules, PII considerations, anonymisation and usage policies in partnership with AI Governance and Compliance.

Platform & Tooling

• Own requirements for dashboards, monitoring tools, model comparison views, anomaly detection alerts, and performance scorecards.
• Design systems that allow product and engineering teams to self‑serve insights about AI model behavior and MCP interactions.
• Partner with engineering to build pipelines that support real‑time and batch analytics.

Cross‑Functional Collaboration

• Collaborate with product owners across LSEG to instrument their AI features consistently.
• Partner with AI Evaluation PM (previous role), Model Risk, GSSR, Legal and Compliance to align telemetry with governance frameworks.
• Work closely with Engineering and SRE teams to drive observability improvements and reliability engineering for AI systems.

Optimisation & Insights

• Identify cost inefficiencies across model and MCP usage, and drive recommendations to improve ROI.
• Surface workflow‑level insights on how customers use AI features—informing product roadmaps.
• Partner with product leaders across divisions to understand how telemetry can improve customer experience, reliability and performance.

Governance & Standards

• Define and maintain AI telemetry standards and best practices across all LSEG divisions.
• Contribute to LSEG’s AI governance and Responsible AI initiatives by providing data‑driven insights on model behavior and user impact.
• Ensure telemetry systems support auditability, compliance and explainability requirements.

Required Skills & Competencies

• Experience in product management with a strong foundation in observability, telemetry, data platforms, monitoring, or SRE / DevOps‑driven products.
• Understanding of LLMs, embeddings, vector search, MCP tools, and AI inference workflows.
• Deep familiarity with logging, tracing, metrics, and event‑based telemetry systems.
• Ability to define data schemas, signal taxonomies, aggregation strategies and data contracts.
• Strong analytical skills and ability to derive insights from large‑scale system telemetry.
• Experience working with senior engineering, data science, risk and governance stakeholders.

Preferred

• Exposure to financial data, analytics systems and enterprise‑scale data workflows.
• Familiarity with BI tools, metrics stores, distributed tracing and monitoring stacks.
• Understanding of cloud infrastructure, serverless runtimes, API gateways and model hosting architectures.
• Ability to drive cross‑functional alignment and establish group‑wide standards.

Measures of Success

Platform Reliability & Performance

• Reduction in AI system error rates, latency spikes and operational incidents.
• Stability and transparency of MCP usage across product lines.
• Quality and coverage of telemetry signals across all AI‑enabled products.

Insight Generation & Adoption

• Adoption of telemetry dashboards and self‑service tools across divisions.
• Number of identified and resolved model or MCP reliability issues driven by telemetry.
• Level of insight generated to influence product prioritisation and AI model choices.

Operational Efficiency & Cost

• Improved cost transparency and cost‑saving impact via telemetry‑driven optimisation.
• Reduction in unnecessary model invocation or inefficient tool usage.

Governance & Compliance

• Contribution to Responsible AI frameworks through robust, auditable telemetry.
• Improved explainability and traceability of AI features for internal/external stakeholders.

Career Stage

Manager

Equal Employment Opportunity Statement

We are proud to be an equal opportunities employer. We do not discriminate on the basis of any protected characteristic and we accommodate religious practices, mental health or physical disability needs. Candidates are required to read the privacy notice regarding personal information.

#J-18808-Ljbffr…