Original listing text, shown exactly as published by the company.
Responsibilities
- Establish and maintain automated frontend testing.
- Build API and integration test coverage for routers and database flows.
- Design and implement E2E tests with Playwright for critical user flows.
- Create stable mocking and fixture strategies (MSW, pytest fixtures, predefined test data).
- Configure quality gates in CI for linting, unit/integration/E2E tests, code coverage, and reporting.
- Extend coverage into non-test UI and workflow areas and prevent regressions.
- Build an LLM evaluation discipline with golden datasets and measurable quality gates.
- Track reliability of model outputs, detect drift, and condition releases on quality/stability thresholds.
- Improve test maintainability, eliminate flakiness, and optimize runtimes.
Qualifications
3+ years in QA automation on production systems.
- Strong Python testing skills with pytest.
- Strong frontend testing skills.
- Experience building and maintaining E2E suites with Playwright/Selenium.
- Experience in API testing (REST, schema validation, authentication/error scenarios).
- Experience with SQL/PostgreSQL testing and test data management.
- Experience with CI/CD pipelines (GitHub Actions, Azure Pipelines or Jenkins).
- Practical understanding of testing non-deterministic outputs of LLM models.
- A plus
- Experience in performance and benchmark testing.
- Experience in WebSocket testing.
- Knowledge of security and dependency scanning.
- Experience in legal-tech, fintech or other regulated domains.
- Familiarity with LLM quality metrics (schema validity, hallucination tracking, consistency).
- Tech Stack
- Backend Testing: pytest, pytest-asyncio, coverage.py
- E2E: Playwright
- CI/CD: GitHub Actions…
