In modern software development, Build Verification Testing (BVT) and Regression Testing are essential to maintaining application stability and functionality. Traditional manual testing methods for these cycles are time-consuming, error-prone, and delay feedback to developers. To address these challenges, Fortude automated both the BVT and Regression Testing processes using the Fortest automation framework for the apparel manufacturer. This implementation significantly reduced testing time, improved test coverage, and enhanced defect detection, ultimately leading to faster release cycles, cost savings, and improved software quality.

This blog revisits each phase of the project and highlights the benefits delivered at every stage of the automation journey.

Pre-automation challenges

The manual testing process presented several bottlenecks:

• Time-consuming cycles: Each manual testing cycle required ~6 hours for BVT and ~16–18 hours for regression testing. With multiple change requests (CRs) being implemented and several releases happening each week, this manual effort would multiply significantly—making the process increasingly unsustainable without automation.
• Limited test coverage due to time constraints.
• Prone to human error, leading to inconsistent results.
• Delayed feedback, impacting release timelines.
• Inefficient use of QA resources, who were tied up with repetitive tasks.

These limitations made a strong case for automating both BVT and regression testing using the Fortest framework.

BVT automation: Phased rollout

BVT automation was the first step to catch defects early after each build. The automated suite checked core application functionalities and provided fast developer feedback.

To ensure a smooth rollout and maximize critical module coverage, the automation was deployed in four phases. This phased approach enabled incremental testing, quick feedback, and early issue resolution, reducing disruption risks. Each phase focused on specific modules, prioritizing critical functions first. By the final phase, all essential modules had full BVT coverage.

Key highlights of each phase

• Phase 1: Covered core modules in FLO System – achieved 100% BVT coverage, while WMS reached 30% due to complexity. This phase enabled early defect detection.
• Phase 2: Expanded automation to Dashboard and PTS, both at 100% coverage. WMS coverage grew to 60%, improving stability and core functionality testing.
• Phase 3: Fully automated PPS with 100% coverage. WMS reached 80%, boosting test efficiency and reducing manual effort.
• Phase 4: Completed automation for Inspection, PPS, and WMS, each achieving 100% coverage. Enabled full automation of the build verification cycle.

Phased rollout results

• Quick feedback: Faster feedback loops supported quicker development iterations.
• Risk mitigation: Gradual rollout helped catch and fix automation issues early.
• Continuous improvement: Insights from each phase refined scripts and test coverage, enhancing reliability and efficiency.

Regression automation

Regression testing ensures new changes don’t break existing features. Initially manual and error-prone, the process was automated to cover 67 test cases.

Regression automation was rolled out in four phases, each targeting specific modules. This phased approach allowed for gradual delivery of scripts, improved test coverage, and better use of resources. It also enabled ongoing script refinement and steady gains in execution speed and accuracy.

Key highlights of each phase

Phase 1 – Inspection, SMS

• Modules: Inspection and SMS prioritized for core functionality checks.
• Effort: 140 hours over 18 days.
• Outcome: Inspection achieved 100% automation; SMS reached 30%. Laid the groundwork by automating simpler test cases.

Phase 2 – PPS (Master PO Planning)

• Modules: PPS – Master PO Planning.
• Effort: 154 hours over 20 days.
• Outcome: 100% automation of this complex area, improving test accuracy and early issue detection.

Phase 3 – PPS (Planning), Dashboards, WMS

• Modules: PPS (Planning), Dashboards, WMS.
• Effort: 203 hours over 26 days.
• Outcome: Full automation for PPS and Dashboards; WMS reached 60%. Expanded test scope and covered more intricate workflows.

Phase 4 – PPS (Cutting, Packing), WMS (Production Requests), PTS

• Modules: Final critical modules including PPS (Cutting, Packing), WMS (Production Requests), and PTS.
• Effort: 189 hours over 24 days.
• Outcome: Achieved 100% automation across remaining modules, finalizing the regression suite and minimizing manual testing.

Results of the rollout

• Full coverage: Complete automation of regression tests by end of Phase 4.
• Efficient resourcing: Resources focused where needed most, with complexity increasing across phases.
• Risk reduction: Early detection of issues reduced downstream risks.
• Faster feedback: Each phase improved the speed and quality of developer feedback.
• Improved quality: 100% automation ensured reliable, efficient testing with fewer errors and faster releases.

Benefits of automation

1. Time savings

• BVT execution: Cut from 6 hours (manual) to 30 minutes (automated) — saving 5.5 hours per build.
• Regression testing: Reduced from 16–18 hours to 8–10 hours per cycle — saving up to 1.5 workdays each cycle.

2. Off-hours test execution

• Automation runs overnight via Azure DevOps Pipelines.
• Test results are ready by the start of the workday, enabling quicker issue resolution and faster development cycles.

3. Cost savings

• Manual cost benchmark: QA Engineer salary = $2,000/month → $12.50/hour.
• Annual manual testing load:
  • BVT: 312 hours/year
  • Regression: 832–936 hours/year
• Automation significantly cuts this effort, translating to substantial cost savings in recurring QA operations.

Recurring automation execution costs: The annual cost of executing the automation suite, using Azure Pipelines, is $504/year, which is a significantly lower expenditure compared to manual testing.

Cost savings calculation:

Annual Savings: $15,096 – $504 = $14592

The automation of BVT and Regression Testing results in annual savings of 90%, directly attributed to the use of automation.

The automation of Build Verification Testing (BVT) and Regression Testing through the Fortest framework has led to impressive improvements in speed, cost-efficiency, and overall quality. BVT now completes in just 30 minutes, down from 6 hours manually, while regression testing is reduced to 8–10 hours instead of the usual 16–18. This has shaved off 1.5 days of testing per release. Additionally, the automation has freed up QA resources to focus on more valuable tasks like exploratory testing, ultimately enhancing product quality. With immediate feedback after every build, even outside of working hours, the development cycle is significantly faster. Overall, the implementation of Fortest has not only accelerated release times but also optimized resources and substantially reduced costs.

If you want to know how the Fortest framework can benefit your business, get in touch with our consultants.

Terminology guide

SMS – Style Management System

WMS – Warehouse Management System

PTS – Production Tracking System

PPS – Production Planning System