Director, Search & AI Evaluation

About the Team

Elsevier’s Search & Evaluation organisation drives AI‑powered discovery by building intelligent retrieval, ranking, and evaluation systems that power trusted scientific and healthcare experiences. The team advances search relevance, retrieval quality, experimentation frameworks, and AI evaluation capabilities across Elsevier’s next‑generation AI platforms and products.

About the Role

As Director of Data Science – Search & Evaluation, you will lead the strategic direction, technical vision, and organizational growth of our Search & Evaluation function. You’ll focus on defining and scaling evaluation frameworks, search relevance methodologies, retrieval optimisation strategies, and AI quality measurement systems across Elsevier’s AI‑powered discovery experiences.

Key Responsibilities

Search, Retrieval & AI Quality Strategy

• Define and drive the long‑term strategy for search relevance, retrieval evaluation, ranking optimisation, and AI system quality.
• Lead initiatives to improve search relevance, ranking quality, semantic retrieval, vector search, and retrieval‑augmented generation (RAG) systems.
• Mitigate AI grounding and hallucination issues, driving user discovery and engagement outcomes.
• Establish scalable evaluation methodologies for search, retrieval, recommendation, and LLM‑powered systems.
• Guide experimentation and optimisation across lexical, semantic, hybrid, and AI‑assisted retrieval architectures.
• Partner with Product and Engineering leadership to align search and AI investments with customer and business priorities.
• Influence technical direction for retrieval systems, evaluation infrastructure, and AI quality frameworks across platforms.

Evaluation & Experimentation Leadership

• Define and operationalise evaluation frameworks for search and generative AI systems, including IR metrics (NDCG, recall, precision) and LLM/RAG evaluation methodologies.
• Develop grounding and faithfulness evaluation, human evaluation, and annotation strategies.
• Design online experimentation and A/B testing, establishing best practices for offline benchmarking and reproducible evaluation workflows.
• Build scalable processes for benchmark creation, annotation quality, evaluation governance, and performance reporting.
• Drive rigorous, evidence‑based decision‑making across AI and search initiatives.
• Champion responsible AI practices focused on quality, reliability, trust, and measurable user impact.

Organisational & Cross‑Functional Leadership

• Lead, mentor, and grow a high‑performing team of Data Scientists and Analysts.
• Create a culture of scientific rigor, accountability, collaboration, innovation, and continuous learning.
• Partner closely with Product Managers, Engineers, UX Researchers, and Applied AI teams to deliver impactful AI capabilities.
• Translate complex technical findings into clear business insights and strategic recommendations for senior stakeholders and executive leadership.
• Help define organisational priorities, road‑maps, and operating models for Search & Evaluation initiatives.
• Drive alignment across cross‑functional teams operating in fast‑moving, ambiguous AI environments.
• Contribute to long‑term AI and discovery strategy across Elsevier platforms.

Qualifications

• Master’s or PhD in Computer Science, Data Science, Machine Learning, Information Retrieval, Statistics, NLP or a related quantitative field.
• 10+ years of experience in Data Science, Machine Learning, Information Retrieval, Search Relevance, Evaluation Systems, or Applied AI.
• Significant experience leading and scaling high‑performing technical teams in complex, cross‑functional organisations.
• Deep expertise in search relevance and ranking systems, semantic search, retrieval‑augmented generation, evaluation methodologies for IR and generative AI, experimentation frameworks and A/B testing.
• Strong experience with vector retrieval and hybrid search architectures, LLM evaluation and AI quality measurement, embeddings, reranking, and retrieval orchestration.
• Experience with evaluation datasets, benchmarking, annotation workflows, and advanced programming skills in Python.
• Familiarity with modern AI/ML frameworks and tooling (PyTorch, Hugging Face, LangChain, LangGraph, Haystack).
• Experience working with large‑scale datasets, distributed data/ML platforms, and production AI systems.
• Strong understanding of statistical analysis, experimentation design, and evaluation science.
• Excellent communication and stakeholder management skills, including experience influencing senior leadership.
• Demonstrated ability to balance strategic leadership with pragmatic execution in rapidly evolving AI environments.

Preferred Qualifications

• PhD preferred in Computer Science, Machine Learning, NLP, Information Retrieval, Statistics, or a related field.
• Experience leading search, ranking, recommendation, or AI evaluation organisations at scale.
• Experience building evaluation systems for LLM‑powered applications and AI assistants.
• Familiarity with scientific, biomedical, or scholarly datasets and workflows.
• Experience with knowledge graphs, ontologies, or semantic enrichment systems.
• Experience with production ML systems, MLOps, and AI governance practices.
• Publications or applied research contributions in NLP, IR, search, recommendation systems, or generative AI.
• Experience building AI systems in high‑trust, regulated, or content‑rich domains.

Benefits

• Comprehensive Pension Plan
• Home, office, or commuting allowance
• Generous vacation entitlement and option for sabbatical leave
• Maternity, Paternity, Adoption, and Family Care leave
• Flexible working hours
• Personal Choice budget
• Internal communities and networks
• Employee discounts
• Recruitment introduction reward
• Employee Assistance Program (global)

Equal Opportunity Employer

Elsevier is an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law. USA Job Seekers: EEO Know Your Rights.

#J-18808-Ljbffr…