Automation Framework

The Clients large aggregated data is backed by thousands of calculators and a strong financial search engine. Major part of the entire system is developed in association with Monocept over a strong partnership since 5 years. Stringent quality checks are required for any code/configuration changes implemented in this complex real-time financial application. Any issues in the application may directly result in huge revenue loss.

They have developed the following three kinds of tracking mechanism which is their prime business revenue model.

• Cost per click (CPC) model follows pay per view basis
• Omniture for traffic/trend monitoring keeps track of the incoming request types like Market, Product, Amount, No of months, Request origin (CO BRAND) Yahoo, AOL and so on.
• Advertisements model hosts ads of various products in their main site and other satellite sites. These ads are chargeable on the cost per click model.

The pain point with the above revenue model here is that a simple functional change could break the entire revenue system (tracking mechanism). This could impact wrong parameters for CPC and Omniture calls getting stored in the DB, tracking of the co brand listings getting missed and the existing call tracing mechanism having browser compatibility issues. So all these had to be manually tracked by QA by running huge set of more than 10000 tests which almost costs 3 – 4 minutes for running each test case across all browsers. This manual effort requires lot of human interaction and concentration and sometimes results in missing the important tests due to tight test schedules.

There are existing Coded UI test developed using Visual Studio Team Test by the team had following severe limitation of:

• Supporting only IE browser
• No built in mechanism for storing test results

Hence a robust functional automation framework needs to be designed to overcome these challenges which will directly reduce the risk of huge revenue loss. Being a long term partner, they looked at Monocept to solve this humongous problem through right automation framework.

Automation scripts need to run on different browsers like Firefox and Chrome for which a new cost effective tool needs to be evaluated.
The tool also needs to minimize the manual intervention from the QA engineers. Hence the Test Automation exercise must:

• Be a seamless automation solution for executing tests across all browsers.
• Be easy to execute in batches without requiring manual intervention.
• Enable executing the regression suits of > 10000 test cases without slipping the test execution schedule.
• Have minimal learning curve and reduce investments in Training
• Help in better Resource Management.
• Reusing the tested-and-verified functional automation scripts generated in Visual Studio Coded UI for IE will save time rather than reinventing the wheel & developing an automation framework from scratch. Hence the scripts for supporting other browsers need to be generated. These new scripts have to be compatible with Visual Studio IDE to seamlessly integrate and execute all the scripts in one batch.

Monocept investigated the problem and limitations of Visual Studio of being able to run Automation only in IE and no mechanism for storing test results. It came up with a simple solution to create a customized testing framework to capture and compare the Ad call values from DB with the new results. Selenium was used in Integrated mode with VS to extend running existing tests on other browsers as well. The Reusability advantage, Compatibility with VS IDE and Training needs are all met by generating scripts using Selenium web driver.

Selenium web driver requires installing drivers for each browser separately to be able to generate scripts in C Sharp. This helps in being compatible with executing the scripts on Visual Studio IDE.

Proof of Concept is prepared for automating the initial smoke tests which gained the confidence of the Client.

The POC was extended to automate all > 10000 regression test cases using this approach.

Monocept’s technical expertise in finding simple yet cost-effective solutions for complex problems and experience in core engineering practices gave an edge and confidence to client that Monocept will beat the expectations and deliver the results.

Monocept has come up with two solutions to the client’s challenges stated above.
The first solution was creating a custom testing framework which

• Takes care of capturing all Http requests using fiddler core DLL, filtering Omniture/CPC , Ad calls and writing it to the local xml file and then
• Parse the xml file and create respective Omniture/CPC and Ad objects and
• Provide methods to get corresponding values from database and
• Storing test results into an xml file and uploading it into TFS.

And the second solution was to use the selenium based coded UI testing.

• The scripts generated for IE using VS Coded UI test are reused to save automation time. Monocept generated scripts for all browsers using the Selenium IDE and exported the same to Visual Studio IDE for execution.
• The scripts generated using VS coded UI test and new scripts generated using Selenium web driver are integrated to run test in batches, without manual intervention.
• Selenium web methods are extensively utilized to enhance the automation capabilities like automatic screenshot capture which executing the scripts.
• Configuration changes in Selenium web driver scripts helped in overcoming the challenges of extending the support to cover Safari browser too.
• With this solution it was easy to integrate test cases with Microsoft test manager and TFS.

Technologies/Tools Used:

i. Visual Studio IDE for Coded UI automation
ii. Team Foundation Server
iii. Selenium web drivers for Chrome and Firefox
iv. Dot Net Framework 4.0
v. Fiddler tool