Senior Test Automation Engineer 100% (f/m/d)

Experteer Overview

You drive reusable patterns, contract tests and CI/CD integration, shaping how the bank validates complex AI-driven capabilities at scale. This is a chance to influence testing strategy in a regulated wealth-management environment and deliver auditable, resilient test outcomes.

• Define and evolve the technical test automation approach and framework architecture for the ML & AI ART, aligned with the Bank's Test Strategy and Test Policy
• Design reusable, scalable test automation patterns (page objects, API clients, test data builders) for adoption across squads
• Analyze requirements, features and stories for testability during PI Planning, backlog refinement and iteration planning
• Derive test cases from technical and risk analysis (functional and non-functional requirements) selecting appropriate techniques based on risk and ROI
• Automate identified test cases using Python-based frameworks (Playwright-Python UI, requests + pytest APIs, Behave or pytest-bdd for BDD)
• Design and implement AI/ML-specific test cases: evaluation pipelines for LLM outputs
• Build and maintain contract tests (e.g. Pact) for APIs and microservices boundaries
• Integrate automated tests in GitLab CI/CD pipelines and manage test executions across environments
• Leverage Docker and Kubernetes to provision isolated, reproducible test environments
• Plan, schedule and trigger automated test executions across DEV/INT/UAT/pre-PROD, including regression and smoke tests
• Monitor execution health, manage flaky tests and maintain low false-positive rates
• Triage results, raise Jira defects with evidence, and communicate quality signals to stakeholders
• Produce execution evidence (run metadata, artefacts, reports) for audit and governance
• Contribute to PI Planning, System Demos, Inspect & Adapt and SAFe ceremonies
• Ensure end-to-end traceability from Jira items to tests and results
• Author and maintain Gherkin BDD scenarios linked to acceptance criteria
• Collaborate with Product Owners, Scrum Masters, ML Engineers, MLOps Engineers and Data Engineers
• Align with Test Strategy and reporting requirements, supporting audit-ready artefacts
• Support root cause analysis using logs and traces
• Act as Test Automation Ambassador within ML & AI ART, guiding automation design and tooling
• Prepare test data (synthetic/anonymised) adhering to confidentiality requirements
• Design AI/ML-specific test datasets where needed
• Own test maintenance and refactoring in response to UI, API, model or prompt changes
• Ensure compliance with access controls and test-environment policies

• Python-based test automation expertise (Playwright-Python UI, Behave or pytest-bdd for BDD, requests + pytest for API testing)
• Design and ownership of test automation frameworks with reusable utilities
• CI/CD experience integrating automated tests (ideally GitLab)
• Distributed test execution, parallelisation, flaky-test management and modern reporting (Allure, pytest-html)
• Git and clean code practices; strong code review culture
• Docker knowledge; familiarity with Kubernetes basics
• Exposure to AI/ML testing or strong motivation to develop this expertise (LLM evals, RAG, agentic workflows)
• Understanding of API design, microservices, event-driven architectures and auth layers
• SAFe and DevOps familiarity; Agile Release Train experience a plus
• Jira for story/feature tracking and test management (Xray, Agile Hive)
• End-to-end thinking across user journeys, data flows and system boundaries
• Audit-ready artefacts, traceability and test data handling in regulated environments
• Strong communication, collaboration and ownership with minimal supervision
• Bachelor or Master in Computer Science/Engineering
• Minimum 5-7 years in Test Automation with hands-on Python and framework design
• ISTQB Foundation or equivalent certification (required)
• SAFe SP/SSM or equivalent, or DevOps disciplines are a plus
• Spanish language skills a plus

•