# AI - Software Development Engineer in Test (AI - SDET) Source: https://realanalystjobs.com/job/ai-software-development-engineer-in-test-ai-sdet-jobgether-b94a6699 AI - Software Development Engineer in Test (AI - SDET) at Jobgether, an on-site role based in India. Posted on 11 November 2026, read from Jobgether's own careers page (Lever), which RealAnalystJobs re-checks daily and removes the role when it closes. Updated 15 November 2026. ## Key facts - Company: [Jobgether](https://realanalystjobs.com/companies/jobgether) - Location: India - Work mode: On-site - Posted: Nov 11, 2026 - Employment type: Full time - Source: Jobgether's careers page (Lever), re-checked daily Skills: [SQL](https://realanalystjobs.com/skills/sql), [Python](https://realanalystjobs.com/skills/python), AWS, Azure ## About this role This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an AI - Software Development Engineer in Test (AI - SDET) based in India. As an AI Software Development Engineer in Test, you will play a key role in redefining quality engineering for next-generation AI-powered applications and autonomous systems. You will go beyond traditional software testing to evaluate, validate, and improve large language models (LLMs), AI agents, copilots, chatbots, and intelligent workflows. You will design scalable automation frameworks and AI evaluation systems to ensure the reliability, safety, accuracy, and performance of AI-driven solutions. Working closely with Engineering, AI/ML, Product, Platform, and DevOps teams, you will contribute to the development of robust, cloud-native applications. You will also help advance AI testing methodologies through experimentation, automation, and emerging technologies. This is an exciting opportunity to shape modern quality engineering practices in an innovative, collaborative, and globally distributed environment. Accountabilities: • AI and LLM Quality Engineering • Design, implement, and optimise testing strategies for AI-powered applications, copilots, chatbots, Retrieval-Augmented Generation (RAG) pipelines, and autonomous AI agents. • Develop AI evaluation (Evals) frameworks to assess hallucinations, groundedness, toxicity, bias, response quality, contextual relevance, prompt regression, consistency, and safety. • Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics. • Create synthetic and benchmark datasets to support comprehensive AI testing scenarios. • Test multi-agent orchestration workflows, including agent memory, context retention, planning, and execution. • Validate Model Context Protocol (MCP) integrations and interoperability between AI models and external tools. • Apply Context Engineering, Prompt Engineering, and prompt optimisation techniques to improve AI reliability and output quality. • Evaluate AI system performance, including latency, token consumption, cost efficiency, throughput, and scalability. • Automation Engineering • Develop, maintain, and scale UI and API automation frameworks using Python, preferably with Robot Framework and tools such as Playwright, Selenium, or Cypress. • Automate testing across REST APIs, GraphQL APIs, event-driven systems, streaming applications, webhooks, microservices, and Micro Frontend (MFE) architectures. • Build reusable automation libraries, testing utilities, and AI-assisted quality engineering workflows. • Integrate automated testing into CI/CD pipelines and implement quality gates to improve software delivery reliability. • AI Engineering and Platform Validation • Validate RAG architectures by assessing document chunking strategies, embedding quality, retrieval accuracy, re-ranking, and vector search relevance. • Work with vector databases and search technologies, including Pinecone, Weaviate, ChromaDB, Qdrant, and FAISS. • Build, test, and validate AI agents using frameworks such as LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel, and OpenAI SDKs. • Test advanced AI capabilities, including tool calling, function calling, agent memory, planning and execution, multimodal workflows, and structured outputs. • DevOps, Observability, and Reliability • Build and maintain automated quality workflows integrated with CI/CD platforms such as GitHub Actions, Jenkins, GitLab CI/CD, and Azure DevOps. • Work with Docker and Kubernetes environments to validate application deployment, scalability, and reliability. • Perform cloud-based testing and deployments across AWS, Azure, Oracle Cloud Infrastructure (OCI), and Google Cloud Platform (GCP). • Validate distributed systems, focusing on reliability, resilience, and performance under different operating conditions. • Monitor AI system behaviour using observability, logging, and distributed tracing tools. • Promote shift-left testing practices and contribute to continuous improvement in quality engineering standards. Requirements: • Technical Experience and Core Skills • Strong programming skills in Python, with the ability to develop and maintain robust automation frameworks and testing solutions. • Solid understanding of software testing principles, quality engineering methodologies, and modern software development practices. • Hands-on experience with AI and LLM testing, Prompt Engineering, Context Engineering, AI evaluation frameworks, RAG systems, and AI agents. • Experience building and maintaining CI/CD pipelines and working with Docker, Kubernetes, and cloud platforms such as AWS, Azure, OCI, or GCP. • Strong expertise in API testing and automation, microservices testing, and UI automation. • Experience with SQL, database testing, and data validation. • Strong debugging, analytical, and troubleshooting abilities, with a structured approach to solving complex technical challenges. • Familiarity with Agile and Scrum methodologies and experience collaborating within cross-functional engineering teams. • Preferred Qualifications and Additional Skills • Experience with Robot Framework and Playwright for automated testing. • Knowledge of AI development frameworks and ecosystems, including LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel, and Model Context Protocol (MCP). • Practical experience using AI development assistants and platforms such as Cursor, GitHub Copilot, Claude, Gemini, and OpenAI APIs. • Familiarity with vector databases, embeddings, and semantic search technologies. • Exposure to MLOps workflows and machine learning lifecycle testing. • Knowledge of performance testing, security testing, and AI governance and compliance requirements. • Experience with contract testing, chaos engineering, or resilience testing. • Familiarity with accessi Description from Jobgether's public careers feed, reproduced so you can read the role here. Apply on the company's own site; Real Analyst Jobs never submits anything for you. ## How to apply Open [this role on Real Analyst Jobs](https://realanalystjobs.com/job/ai-software-development-engineer-in-test-ai-sdet-jobgether-b94a6699) and use Apply now there to reach the employer's application. The listing is re-checked against Jobgether's careers page daily and removed when it closes. ## Jobgether 198 open analyst roles on Real Analyst Jobs. [More roles at Jobgether](https://realanalystjobs.com/companies/jobgether) ## More like this ### Other open roles at Jobgether - [Blockchain Intelligence Analyst, Transnational Crime Organizations](https://realanalystjobs.com/job/blockchain-intelligence-analyst-transnational-crime-organizations-jobgether-a69032ed), France (On-site) - [Blockchain Intelligence Analyst, Transnational Crime Organizations](https://realanalystjobs.com/job/blockchain-intelligence-analyst-transnational-crime-organizations-jobgether-874a2e71), Portugal (On-site) - [Senior Reimbursement Analyst- Regulatory and Reporting](https://realanalystjobs.com/job/senior-reimbursement-analyst-regulatory-and-reporting-jobgether-50b894e7), USA (On-site) - [Blockchain Intelligence Analyst, Transnational Crime Organizations](https://realanalystjobs.com/job/blockchain-intelligence-analyst-transnational-crime-organizations-jobgether-bbb64ea5), UK (On-site) - [Senior Data Business Analyst / Product Owner (Contractor)](https://realanalystjobs.com/job/senior-data-business-analyst-product-owner-contractor-jobgether-228735bb), Germany (On-site) - [Data analyst jobs in India](https://realanalystjobs.com/jobs/data-analyst-jobs-in-india) - [Analyst job market report](https://realanalystjobs.com/analyst-job-market) - [What analyst roles pay](https://realanalystjobs.com/blog/data-analyst-salaries) ## Sources and links - [This role on Real Analyst Jobs](https://realanalystjobs.com/job/ai-software-development-engineer-in-test-ai-sdet-jobgether-b94a6699) - [Jobgether on Real Analyst Jobs](https://realanalystjobs.com/companies/jobgether) - [All analyst jobs](https://realanalystjobs.com/jobs)