16 nov
|
Web.com Canada
|
Buenos Aires
16 nov
Web.com Canada
Buenos Aires
This role goes beyond traditional QA; the adecuado candidate should be capable of validating, evaluating, and improving AI-powered systems, agents, copilots, and LLM-driven workflows while building scalable automation frameworks and intelligent quality solutions. You will work closely with Engineering, AI/ML, Platform, Product, and Dev Ops teams to ensure reliability, safety, observability, scalability, and quality of modern AI-native applications and Agentic systems. Who Are We.
Newfold Digital is a leading web technology company serving nearly seven million customers globally. Established in 2021 through the combination of leading web services providers Endurance Web Presence and Web.com Group, our portfolio of brands includes: Bluehost, Big Rock, Reseller Club, Crazy Domains, Host Gator, Network Solutions, Register.com, Web.com and many others. We help customers of all sizes build a digital presence that delivers results. With our extensive product offerings and personalized support, we take pride in collaborating with our customers to serve their online presence needs. Key Responsibilities
AI/LLM Quality Engineering
Design, validate, and optimise testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents. Develop AI evaluation (Evals) frameworks for: Hallucination detection Groundedness Toxicity & bias validation Response quality Context relevance Prompt regression testing Determinism & consistency Safety and guardrail validation Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics. Create synthetic datasets and benchmark datasets for AI testing scenarios. Test multi-agent orchestration workflows and agent memory/context retention. Validate MCP (Model Context Protocol) integrations and tool interoperability.
Work on Context Engineering, Prompt Engineering, and Prompt Optimisation strategies. Evaluate latency, token usage, model cost optimisation, throughput, and scalability. Automation Engineering
Develop and maintain scalable UI/API automation frameworks using: Python (mandatory)Robot Framework (preferred)Playwright/Selenium/Cypress Automate testing for:REST APIsGraphQL APIsEvent-driven systems Streaming systems Webhooks MicroservicesMFE (Micro Frontend) architectures Build reusable automation libraries, utilities, and AI-assisted testing workflows. Integrate automation into CI/CD pipelines and quality gates.AI Engineering & Platform Validation
Validate RAG architectures including: Chunking strategies Embedding quality Retrieval accuracy Re-ranking Vector search relevance Work with vector databases such as: Pinecone Weaviate ChromaDBQdrantFAISSBuild or validate AI agents leveraging: Lang Chain Lang Graph CrewAIAuto GenSemantic Kernel OpenAI SDKsValidate AI workflows involving: Tool calling Function calling Agent memory Agent planning/execution Multi-modal AI systems Structured outputs DevOps, Observability & Reliability
Build and maintain quality workflows integrated with: Git Hub Actions Jenkins GitLab CI/CDAzure Dev OpsWork with Docker and Kubernetes environments. Perform cloud-based testing and deployments across AWS/Azure/OCI/GCP. Validate distributed systems reliability and resiliency.
Monitor AI system behaviour using observability platforms/logging/tracing tools. Contribute to shift-left and quality engineering best practices. Required Skills
Must Have
Strong programming experience in Python Strong understanding of Software Testing & Quality Engineering principles Hands-on experience with:AI/LLM testing Prompt Engineering Context EngineeringAI EvalsRAG systemsAI Agents Experience with:CI/CD pipelines Docker & Kubernetes Cloud platforms (AWS/Azure/OCI/GCP)API testing & automation Microservices testingUI automation Experience with SQL and data validation Strong debugging, analytical, and troubleshooting skills Experience working in Agile/Scrum environments Preferred / Good to Have
Experience with Robot Framework Experience with Playwright Knowledge of: Lang Chain Lang Graph Llama Index CrewAISemantic KernelMCP (Model Context Protocol)Hands‑on experience with: Cursor GitHub Copilot Claude Gemini OpenAI APIsExperience with vector databases and embeddings Exposure to MLOps workflows and ML lifecycle testing Knowledge of performance testing and security testing Experience validating AI governance/compliance requirements Experience with contract testing, chaos testing, or resilience engineering Familiarity with accessibility testing and cross-browser/device testing Exposure to synthetic monitoring and production observability What We're Looking For
Engineers who actively experiment with AI tools and build side projects, prototypes, PoCs, or open-source contributions. Strong problem solvers who can combine traditional QE practices with modern AI-native engineering approaches. Passion for innovation, developer productivity, automation acceleration, and intelligent testing. Ability to rapidly learn emerging AI frameworks, tools, a
#J-18808-Ljbffr
📌 AI - Software Development Engineer in Test (AI - SDET) (Buenos Aires)
🏢 Web.com Canada
📍 Buenos Aires