We're looking for a QA Engineer with experience testing AI-powered applications .

The ideal candidate has hands‑on experience evaluating non-deterministic AI systems (LLMs, AI agents, and RAG applications) and building or maintaining evaluation frameworks (eval engineering) to measure quality, detect regressions, and improve AI system performance.

Sr. QA/Eval Engineer We're looking for a QA Engineer with experience testing AI-powered applications .

The idóneo candidate has hands‑on experience evaluating non-deterministic AI systems (LLMs, AI agents, and RAG applications) and building or maintaining evaluation frameworks (eval engineering) to measure quality, detect regressions, and improve AI system performance.

What are the main responsibilities of this role? Create detailed, comprehensive, and well‑structured test plans and test cases.

Review client needs and technical specifications with the development team to provide timely and meaningful feedback.

Estimate, prioritize, plan,



and coordinate testing activities.

Identify, record, document thoroughly, and track bugs.

Design, develop, and execute automation scripts (including Playwright-based automation).

Perform thorough regression testing when bugs are resolved.

Track quality assurance metrics and reports.

Design and maintain evaluation suites (evals) to validate AI-powered features and detect regressions in model behavior.

Test non‑deterministic AI systems, including LLM‑based features, AI agents, and RAG applications, using qualitative and quantitative evaluation methods.

Analyze AI outputs, identify failure patterns (hallucinations, instruction‑following issues, grounding, etc.), and collaborate with engineers to improve system quality.

Stay up to date with new testing tools, AI evaluation technique

📌 Sr. QA/Eval Engineer (Argentina)
🏢 Rootstrap
📍 Argentina