ELEKS Quality Assurance Office is looking for a Senior Automation Test Engineer (AI/LLM) in Argentina

ABOUT PROJECT

Our customer is among the top 5 health insurance companies in the US. They are dedicated to expanding access to high-quality, cost-effective healthcare.

We are looking for an experienced Automation Test Engineer to join the project engagement and work embedded within the client’s delivery team, focusing on testing AI-powered applications, including Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) solutions.

REQUIREMENTS

Experience testing LLM/RAG behavior, including intent routing, retrieval quality, prompt safety, and evaluation/regression testing Experience in test automation using Python and PyTest Experience testing REST APIs and FastAPI-based applications Experience with SQL and Databricks data validation Experience testing Azure AI Search Experience testing Azure OpenAI APIs Experience with LLM intent-routing validation Experience with RAG retrieval-quality testing Experience with prompt-injection and PII-leakage testing Experience with groundedness, hallucination, and citation validation Experience with RAGAS or DeepEval evaluation Experience with automated regression testing using JSONL datasets Experience with Postman and Newman Experience with Jenkins CI/CD integration Experience with Docker and Git Experience with Application Insights or OpenTelemetry monitoring At least Upper-Intermediate level of English

RESPONSIBILITIES





Develop, execute, and maintain automated tests using Python and PyTest Perform REST API and FastAPI testing Validate data using SQL and Databricks Test Azure AI Search and Azure OpenAI APIs Validate LLM intent routing and RAG retrieval quality Perform prompt-injection and PII-leakage testing Validate groundedness, hallucinations, and citations in LLM-generated responses Conduct LLM evaluations using RAGAS or DeepEval Perform automated regression testing using JSONL datasets Use Postman and Newman for API testing Integrate automated tests with Jenkins CI/CD Use Docker and Git in test automation activities Monitor applications using Application Insights or OpenTelemetry

WHAT YOU WILL GET WITH ELEKS

Close cooperation with a customer Challenging tasks Competence development Ability to influence project technologies Team of professionals Dynamic environment with low level of bureaucracy

ABOUT

ELEKS is a custom software development company. We deliver value to our clients, thanks to our expertise and experience gained from working as a software innovation partner since 1991.

Our 2000+ professionals located in the Delivery Centers across Eastern Europe and sales offices in Europe and North America, provide our clients with a full range of software engineering services. These include product development, QA, R&D;, design, technology consulting and dedicated teams.

Benefits

undefined

📌 Senior Automation Test Engineer (Argentina)
🏢 Eleks
📍 Argentina