AI / MLOps Engineer

SENIOR AI ENGINEER – RAG & LLM APPLICATIONS

Department: Engineering
Experience: 6+ Years
Relevant LLM Experience: 2+ Years
Employment Type: Full-Time
Location: Hyderabad – Work From Office
Engagement: Product / Enterprise AI Engineering

ABOUT NSTARX

NStarX is an AI-first, cloud-first engineering services company focused on building production-ready technology solutions that deliver measurable business value. We work across AI/ML, cloud engineering, automation, data engineering, and software engineering to solve complex enterprise problems.

At NStarX, we focus on building reliable, scalable, and production-grade AI solutions rather than experimental prototypes.

ABOUT THE ROLE

This is a core AI engineering and product-build role. You will own the retrieval and grounding layer, document and presentation generation, and the orchestration layer that connects these capabilities to an enterprise-governed LLM/model gateway.

The primary objective of this role is output correctness and reliability. Every generated claim must be supported by retrieved evidence and linked to its source. The system must be designed to fail closed — when sufficient evidence is unavailable, it should withhold the output rather than generate an unsupported statement.

You will work across RAG, LLM orchestration, evaluation, evidence retrieval, document generation, and enterprise AI integration to build reliable AI-powered products for production use.

RAG & GROUNDING:
Design, build, and continuously improve Retrieval-Augmented Generation (RAG) pipelines over a PostgreSQL-backed knowledge store. Develop effective document chunking, indexing, embedding, retrieval, filtering, and reranking strategies. Improve retrieval quality through experimentation with search strategies, metadata, hybrid retrieval, and reranking. Implement robust citation and evidence-traceability mechanisms so generated claims can be linked back to their supporting sources. Implement entitlement-aware retrieval and access controls, including row-level security where applicable.

EVIDENCE & SOURCE INTEGRATION:
Integrate real-time evidence services using external public and licensed data sources. Ensure evidence is appropriately dated and includes reliable source attribution. Design workflows that distinguish between retrieved evidence, generated content, and unsupported claims. Handle source conflicts, stale information, missing evidence, and insufficient retrieval gracefully.

LLM & AGENT ORCHESTRATION: Build production-grade orchestration workflows connecting retrieval, reasoning, validation, and generation components. Integrate with an enterprise-governed model gateway/proxy rather than directly calling individual model providers. Design workflows within defined token, latency, and cost budgets. Contribute to multi-agent or persona-based orchestration where required. Implement appropriate guardrails around model inputs, outputs, context, and tool usage.

FAIL-CLOSED QUALITY & EVALUATION:
Build a fail-closed validation engine consisting of named, independently reportable pass/fail checks. Ensure validation failures can block delivery of unsupported or insufficiently grounded content. Develop automated evaluation harnesses for RAG and LLM workflows.
RESPONSE LATENCY: Establish repeatable evaluation datasets and regression testing for AI behavior.

DOCUMENT & PRESENTATION GENERATION:
Build generators that transform retrieved evidence into predefined and validated output contracts. Generate structured business documents and presentations while maintaining source and workflow metadata.
PDF: Work with libraries such as python-pptx, python-docx, ReportLab, or equivalent technologies. Ensure workflow-contract metadata and source attribution are preserved through the generated artifact.

BACKEND & API ENGINEERING:
Develop production-grade APIs and backend services using Python and FastAPI or equivalent frameworks. Design clean service interfaces between retrieval, orchestration, validation, evidence, and document-generation components. Implement appropriate error handling, observability, logging, authentication, authorization, and performance controls. Design services for scalability, reliability, and maintainability.

EXISTING PLATFORM ASSESSMENT:
Contribute to the current-state assessment of an existing AI/LLM codebase. Review existing architecture, implementation quality, automation, evaluation coverage, and test suites. Identify technical debt, reliability gaps, performance issues, and opportunities for improvement. Recommend practical improvements to architecture, testing, evaluation, and production readiness.

DEPLOYMENT & ENGINEERING PRACTICES:
Build and deploy containerized AI services using Docker and cloud/container platforms. Integrate automated testing and deployment into CI/CD pipelines. Establish appropriate unit, integration, API, regression, and AI evaluation testing. Participate in code reviews, architecture discussions, technical design, and engineering best practices.

EXPERIENCE

6+ years of professional software engineering experience. 2+ years of hands-on experience building and shipping LLM-backed applications to production. Strong experience building production systems rather than prototypes, POCs, or experimentation-only projects.

PYTHON & BACKEND ENGINEERING
Strong hands-on expertise in Python. Strong experience developing production APIs using FastAPI or equivalent frameworks. Solid understanding of REST APIs, asynchronous processing, error handling, logging, observability, and service architecture.

RAG & RETRIEVAL
Demonstrated end-to-end experience building production RAG systems.

RETRIEVAL QUALITY MEASUREMENT
Ability to troubleshoot and improve poor retrieval and grounding performance.

LLM & AI ENGINEERING
Hands-on experience with LLM application development and orchestration frameworks. Strong understanding of LLM limitations, hallucination, context windows, token usage, and inference cost. Experience implementing grounding, validation, guardrails, and structured output mechanisms. Ability to design reliable workflows around LLMs rather than relying solely on prompt engineering.

EVALUATION & QUALITY
Demonstrated experience using an evaluation harness in production or serious engineering environments. Ability to define measurable evaluation criteria and discuss tracked metrics such as:

COST PER QUERY
Experience creating evaluation datasets and automated regression tests for LLM applications.

DOCUMENT GENERATION
Experience with programmatic document and presentation generation using technologies such as: python-pptx python-docx

REPORTLAB
Or equivalent libraries/frameworks. Understanding of structured output contracts and document rendering workflows.

DEVOPS & DEPLOYMENT
Experience with Docker/containerised deployments. Experience with CI/CD pipelines and automated testing. Familiarity with cloud-based application deployment and production monitoring.

PREFERRED QUALIFICATIONS

Experience designing multi-agent or persona-based LLM orchestration. Experience integrating with an enterprise model gateway, LLM proxy, or governed AI platform. Experience working within model/token/cost budgets. Strong experience with PostgreSQL, including row-level security or other entitlement-aware data access mechanisms. Experience implementing semantic caching and query optimization. Experience optimizing LLM applications for cost, latency, throughput, and scalability. Experience with real-time web/evidence retrieval and licensed data sources. Experience with AI observability and LLM tracing platforms. Experience with structured generation and schema-constrained LLM outputs. Experience working with enterprise data governance, security, privacy, and access-control requirements.