Python QA Automation Engineer

We’re looking for a Python QA Automation Engineer to join EPAM in London, in an onsite working mode, contributing to an AI-driven enterprise project for one of our clients. This role focuses on exposing internal enterprise tooling to AI agents via Model Context Protocol (MCP), ensuring robust quality engineering and compliance across innovative intelligent automation solutions.As a Python QA Automation Engineer, you will be responsible for developing automated testing frameworks, evaluation pipelines, and quality controls for MCP Components within our AI platform. You will work closely with engineering and product teams in a Classic Agile environment to guarantee the reliability, accuracy, and performance of agent-driven workflows operating in enterprise-scale systems. This position offers a chance to define standards for testing AI models and tools in production-like environments while being a critical part of one of EPAM’s most transformative projects.ResponsibilitiesDevelop automated testing frameworks to validate MCP Servers and related AI systemsDesign and implement evaluation strategies for LLM accuracy, safety, and reliabilityCreate automated tests using Python, Pytest, and BDD frameworksBuild quality gates into CI/CD pipelines to maintain continuous assuranceIdentify and address agentic AI failure modes such as hallucination, latency, and incorrect tool usageCollaborate with engineering, QA, and product teams to define quality metrics and acceptance criteriaContribute to Agile ceremonies, ensuring testing practices align with sprint goalsPrepare detailed reporting on quality outcomes and improvement opportunitiesMaintain documentation for test cases, evaluation pipelines, and validation strategiesRequirementsStrong programming experience in Python applied to test automation and evaluationExpertise in Pytest and familiarity with BDD frameworks such as Behave or CucumberKnowledge of LLM evaluation approaches including RAGAS, DeepEval, or custom pipelinesUnderstanding of common agentic AI issues such as hallucination, tool misuse, and performance bottlenecksFamiliarity with automated testing of AI workflows, distributed systems, or microservices environmentsStrong grasp of Agile delivery methodologies and CI/CD integration for quality checksExcellent communication and problem-solving skills with a focus on accuracy and reliabilityNice to haveExperience with Model Context Protocol (MCP) or other agent orchestration solutionsExposure to observability, monitoring, or logging tools for AI systemsAPI and service integration testing background for multi-layered platformsKnowledge of containerized environments and cloud-native architectureBackground in enterprise AI automation projects or intelligent platform engineeringWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness coverPrivate medical insurance and dental careEmployee Assistance ProgramCompetitive group pension planCyclescheme, Techscheme and season ticket loansVarious perks such as free Wednesday lunch in-office, on-site massages and regular social eventsLearning and development opportunities including in-house training and coaching, professional certifications, and coursesIf otherwise eligible, participation in the discretionary annual bonus programIf otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program#J-18808-Ljbffr…