QA AI Automation Engineer

Job Description:

We are seeking a meticulous and forward-thinking QA AI Automation Engineer to own end-to-end test automation for our enterprise AI platform. In this role, you will design and execute functional, integration, and LLM-specific regression suites to ensure platform reliability and precision. You will also lead performance/load testing, orchestrate release management, and compile validation evidence for critical delivery milestones, working alongside a platform team that handles security scanning and gate enforcement.

Key Responsibilities:

• Automation Suite Ownership: Build, maintain, and scale comprehensive functional, integration, and regression test suites for backend and distributed systems.
• Performance & Load Testing: Design and execute high-scale load and performance testing to ensure system resilience under enterprise workloads.
• CI/CD Integration: Embed automated test execution directly into CI/CD pipelines to support a seamless, continuous integration workflow.
• Accuracy & Validation Design: Develop advanced validation strategies (such as precision/recall-style test designs) tailored for accuracy-critical detection systems.
• Release Management: Oversee release management processes across sprint-based delivery cycles, providing necessary validation evidence packages for production milestones.

MINIMUM REQUIREMENTS

• Backend Automation Experience: Proven background in QA and test automation targeting complex backend services or distributed systems.
• Python Test Frameworks: Expert-level mastery of Python test frameworks (specifically pytest or equivalent) with deep experience in API-level automated testing.
• Strategic Integration Testing: Demonstrated ownership of test strategies and regression suites for highly integrated, data-dependent systems.
• Performance Testing at Scale: Hands-on experience with load and performance testing tools such as k6, Locust, JMeter, or equivalent frameworks.
• CI-Driven Development: Experience managing and configuring automated tests to run natively as steps within continuous integration pipelines.
• Sprint-Based Release Management: Solid understanding of release coordination across fast-paced, sprint-based release structures.
• Statistical Test Design: Experience designing evaluation frameworks for data or detection systems using precision/recall and similar statistical validation metrics.

GOOD TO HAVE

• LLM & Agent Evaluation: Experience implementing LLM/agent evaluation frameworks and regression suites using golden data sets or reference workloads.
• DLP / PII Testing: Background in validating Data Loss Prevention (DLP) or PII detection engines, specifically optimizing for zero-false-negative benchmarks.
• Cloud Environments: Familiarity running tests inside Kubernetes-based environments or participating directly in Disaster Recovery (DR) simulations.
• NFR Verification: Experience pulling together formal non-functional requirement verification metrics and compiling structured acceptance evidence packages for enterprise clients.