Since a last decade or two, therehave been tremendous changes in the software development methodologies. Varietyof tools besides agile development, extreme programming, and test drivendevelopment have changed the philosophy and are still modifying how thesoftware development takes place.
Needless to say, there is a constant need forchange in developing and owning a software development. During the course ofsoftware development project, however, there is a need of continuousmaintenance and growth of the software product as it can be hit by any kind ofunexpected and unplanned changes. One of the biggest challengesthat is faced during software development phase is to handle traceability; abi-directional trace between the requirements and components implementations aswell as between requirements and acceptance tests. In such, manual maintainingof the traceability can be cumbersome and so for providing better andsustainable results, the need for an automated build acceptance test surfaces. Build Verification Test or theBuild Acceptance Test is a set of tests run on every new build to verify thatbuild is testable before it is released to test team for further testing. Thesetests are the core functionality test cases that ensure that the application isstable and can be tested thoroughly. Typically, the entire process isautomated.
The test checks primarily for the project integrity and verifieswhether all the modules are integrated appropriately or not. Due to impropermodule integration, many applications face failure. In simpler words, BVTs are run ondaily builds and if the BVT fails, the build is rejected and a new build isreleased after the bugs are fixed. BVT checks if all the new and modified filesare included in the release, all the file formats are correct, along with everyfile version and language and flags associated with each file.
Keep in mind that the success ofBVT depends on which test cases are included in BVT. For that to happen, onehas to consider the following points:- Include only critical test cases in BVT- All the test cases that are included in BVTshould be stable- All the test cases that are included should haveknown expected results- Ensure that all included critical functionalitytest cases are sufficient for application test coverageTo make the criticalfunctionality test cases inclusion task simple, one needs to communicate withall those involved in the project development and testing life cycle. Settingsome BVT quality standards and meeting these standards by analyzing majorproject features and scenarios will ultimately help ensure BVT success. Some EXAMPLES:A. Testcases to be included in BVT for Text Editor Application1. Test cases for creating the text file2.
Test cases for writing something into the texteditor3. Test case for copy, cut, paste functionality oftext editor4. Test case for opening, saving and deleting textfileFor every major or minor changein the application, these basis critical test cases can be executed and can beeasily accomplished by BVT. The automated BVT suit needs to be maintained andmodified with time. The following will happen once the BVT suite is run:- The result of the BVT execution is sent to allthe email IDs associated with the project- The BVT owner inspects the result of the BVT. Ifthe BVT fails, then the BVT owner diagnose the failure- If the failure cause is the defect in build, allthe relevant info with failure logs is sent to the respective developers- The developer in initial diagnostic replies tothe team about the failure cause. If there is a bug then what will be thebug-fixing scenario- On bug fix, the BVT test suite is executed andif build passes BVT, the build is passed to the test team for further detailfunctionality, performance and other testsThis process keeps on repeatingfor every new build. B.
Presently,Agile is one of the most commonly used methodology in software development lifecycle, with its pre-defined short iteration cycles for 2-3 weeks that deliversquality product. QA teams work with developers to achieve the goal of BVT.Under BVT, the developers set up or create the VMs/Machines for testing, set upthe environment build deployment, verify build and inform the QA team tofurther start testing. If this entire process isautomated, the BVT process will easily save around 3-4 man days on eachiteration besides being reliable and without manual errors. To ensure thesuccess, following steps will be followed:1. Set up Bamboo or Jenkins in a way that thenightly test builds would start at the end of every sprint or manually, whichpulls the latest code from the GIT/SVN, executes unit test cases, generatesbuild and uploads the build on Artifactory. Once the build is uploaded; it willstart the automated BVT plan.
2. The VMs or machines can be generatedautomatically using Amazon Web Service, Dockers or VMWare ESXi Server whichhave their own required software already installed. APIs can be called usingPython, Java or even shell script, whichever is applicable.3. After the VMs are available, ensure that the webapplication server and database software has been installed successfully. Toautomate this option, Ansible can be considered a good option. 4.
With the latest build from Artifactory, deployit on Web Application server with the help of Ansible. Once it is deployedsuccessfully, check if it can access a web page. 5. After the build is deployed, start executing thepre-defined test cases which encapsulate the core functionality of the product.
Automate those test cases as per the requirement using any language or toolsuch as Java, Python, shell script and selenium etc. 6. Add the test step results into Jira directlyfrom the Test script to avoid manual dependencies.7.
Send the final status email to the manual QAteam that the build is stable/not stable for further testing. This is one of the examples toautomate build verification test process in Agile. This is quite understandable thatthe automated acceptance tests play the key role in software developmentprocess. By using tools such as Concordion or Fitnesse bring the developmentprocess to a better level whilst providing several benefits to the developers,clients, business logic writers and quality assurance personnel. The use ofthese tests tools ties acceptance tools into the requirement specifications whichin turn, results in maintenance, keeping requirement specifications in syncwith the system under development and provides inherited traceability betweenthe requirements and acceptance tests. Automated BVT is very effective fordaily as well as long-term builds and helps saves time, cost, and resources ofthe company.