Skip to content Skip to footer
NStarX Inc.
Get in Touch
NStarX Inc.
Let's Talk
Close

Revolutionizing Test Automation for AIML-based Software

December 27, 2024

Business Challenge

A leading Telecom Enterprise’s fraud detection company approached us with challenges with their platform for which our Data Science team developed a multi-output deep learning model capable of predicting multiple aspects of fraud detection etc with advanced technologies and UI.
With the feature complexity and advanced technologies used, running the automated tests as part of smoke tests as well as integration was required to keep up the CICD (Continuous Integration & Continuous Deployment) requirements.
We had to identify the key areas which impact the MLOps, APIs, UI, Data Engineering for automating the tests which will keep running for every code merge (as part of smoke tests) ensure bug-free production releases and integration tests (for early identification of any issues).

Solution

NStarX came up with an Automation Framework to run end-to-end UI tests using the Behavioral-Driven Development (BDD) with Behave using Python Jenkins enabled
  • This is a customized framework using which we can run tests for any combination such as (can call any tests, mentioning required browsers etc).
  • The tests for the UI elements such as Dashboards, Data Science user flows etc are also automated.
The “Test Data Upload” in cloud storage is also taken care of by the automation tests in the framework.
NStarX also came up with a set up API testing tools to cover all the APIs as part of integration testing.
The framework also has the KubeFlow (as part of MLOps) testing capabilities to cover the assets of the KubeFlow.
The user friendly reports are generated as part of the test results in the allure / JUnit.

Technology & Tools

Libraries

PyTest, BEHAVE, ALLURE-BEHAVE, REQUESTS, Selenium.

Test Tools

PyCharm, Python (v 3.10 and above), Postman, Jenkins, Docker

Cloud Platforms

Azure, AWS for continuous integration and testing pipeline automation

Business Outcomes

Test Coverage

85% of the UI manual tests and 100% APIs tests were automated

Release Speed

  • 100% of the smoke tests keep running for every Pull Request to the Production Environment and Integration Testing can be run as and when required.
  • Built and managed pipelines with Jenkins and KubeFlow which reduced the performance delays.
  • Containerization with Docker for Consistent Testing Environments which simplified running tests in the varied environments.
  • Scaling and Orchestrating Tests with Kubernetes and Kubeflow Pipelines which helped us maintain stable test environments across complex workflows.
  • Continuous Monitoring and Real-Time Reporting.

Quality Assurance

Reduced Production defects as most of the defects were deducted at the QA Environment.

Business Impact

Enabled Continuous Integration and Continuous Deployment with Smoke Tests and Integration Tests running in the pipeline as per the requirement.

Sample Report Generated

Sample Test Automation Report

NStarX Test Automation Framework

NStarX Test Automation Framework