Senior AI Software Engineer

Type of Requisition: Pipeline Clearance Level Must Currently Possess: Secret Clearance Level Must Be Able to Obtain: Top Secret Public Trust/Other Required: None Job Family: Data Science and Data Engineering Job Qualifications: Skills: Agile Software Development, Back-End Development, Microservices Architecture Certifications: None Experience: 7 + years of related experience US Citizenship Required: Yes Job Description: GDIT is seeking a highly motivated and experienced Senior AI Software Engineer to support the development, modernization, integration, and sustainment of our on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, VA. The selected candidate will design, develop, and integrate software solutions leveraging Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, document processing technologies, and modern AI inference frameworks. This role requires a strong foundation in software engineering, Python development, API integration, containerization, and AI application architecture. The engineer will help transition existing AI services toward a scalable, containerized architecture while introducing new capabilities that enhance system performance, reliability, security, and user experience. The ideal candidate is a hands-on software engineer who can work across the AI application stack, from backend development and model serving integration to data processing, automated testing, and deployment. The individual will collaborate with AI/ML engineers, data engineers, infrastructure specialists, cybersecurity teams, and government stakeholders to deliver secure, reliable AI capabilities in highly controlled, classified, and air-gapped environments. Key Responsibilities 1. AI Application Development & Integration • Design, develop, test, and maintain Python-based applications, microservices, and APIs supporting an on prem chatbot. • Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems. • Develop custom OpenWebUI Functions, Tools, and integrations to support mission-specific use cases. • Integrate approved LLMs into chatbot using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama. • Develop reusable software components for AI model access, prompt management, response processing, and application workflows. • Collaborate with AI/ML engineers to improve model inference performance, response quality, and application reliability. • Support multimodal AI use cases, including document understanding, optical character recognition, and vision-language model integration. • Develop backend capabilities to support secure enterprise AI workflows. 2. Retrieval-Augmented Generation (RAG) & Document Processing • Design, implement, and optimize RAG workflows for enterprise document search and question answering. • Integrate document processing technologies such as Docling, Tesseract, and vision-language models into automated ingestion pipelines. • Develop document chunking, metadata extraction, embedding generation, and retrieval workflows. • Integrate embedding models and reranking models to improve retrieval accuracy and relevance. • Work with PostgreSQL and pgvector to support vector search, semantic retrieval, and document indexing. • Develop APIs and background processing services for document ingestion and knowledge base management. • Collaborate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information. • Implement testing and evaluation approaches to measure RAG accuracy, retrieval relevance, and answer quality. 3. AI Model Serving & Inference Integration • Integrate and maintain AI inference services using vLLM, Ollama, and compatible model-serving frameworks. • Support the deployment and configuration of approved open-weight LLMs, embedding models, rerankers, and vision-language models. • Develop software integrations using OpenAI-compatible REST APIs. • Assist in optimizing inference configurations, including context length, GPU memory utilization, concurrency, and model serving performance. • Support the transition from existing inference frameworks to improved architectures as technologies mature. • Collaborate with AI/ML engineers to troubleshoot inference errors, model loading issues, and integration failures. • Develop benchmarking and performance-testing utilities for AI workloads. 4. Containerization & Platform Modernization • Support the migration of DS Chat components from traditional deployments into containerized services. • Develop, configure, and maintain container images using Podman or Docker. • Create and maintain Podman Compose configurations defining application services, networking, volumes, and environment variables. • Integrate services such as OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis. • Develop scripts and automation for repeatable installation, deployment, upgrades, and recovery. • Implement health checks, readiness checks, logging, and service dependency management. • Troubleshoot container networking, authentication, storage, GPU access, and service connectivity. • Collaborate with infrastructure and cybersecurity teams to improve deployment consistency and security. 5. Agentic AI & Future Capabilities • Research, prototype, and develop agentic AI capabilities for future DS Chat releases. • Implement controlled tool-calling and multi-step AI workflows. • Develop integrations that allow approved AI models to interact with authorized enterprise APIs and services. • Explore emerging technologies involving AI orchestration, workflow automation, and Model Context Protocol (MCP). • Design secure execution boundaries, permissions, audit logging, and approval mechanisms for AI-driven actions. • Collaborate with stakeholders to evaluate new AI capabilities for mission relevance, security, and operational feasibility. • Implement safeguards