Automation Framework

Designed a robust functional automation framework to overcome existing challenges, thus reducing the risk of huge revenue loss.

Client Background

Our client, a Billion Dollar Company based out of Florida US, having over three decades experience in financial publishing. Today, they are the World’s leading aggregator of financial rate information, offering an unparalleled depth and breadth of financial rate data and content. This company integrates surveys of approximately 4,800 financial institutions in all 50 states in order to provide clear, objective, and unbiased rates to consumers. Their flagship product, provides free rate information to consumers on more than 300 financial products, including mortgages, credit cards, new and used automobile loans, money market accounts, certificates of deposit, checking and ATM fees, home equity loans and online banking fees. Hundreds of print publications depend on them as the trusted source for financial rates and information, including the top 10 newspapers in the country, including The Wall Street Journal, USA Today, The New York Times, The Los Angeles Times and The Boston Globe.

The Clients large aggregated data is backed by thousands of calculators and a strong financial search engine. Major part of the entire system is developed in association with Monocept over a strong partnership since 5 years. Stringent quality checks are required for any code/configuration changes implemented in this complex real-time financial application. Any issues in the application may directly result in huge revenue loss.

CHALLENGE / OBJECTIVE / PROBLEM STATEMENT

They have developed the following three kinds of tracking mechanism which is their prime business revenue model.

Cost per click (CPC) model follows pay per view basis
Omniture for traffic/trend monitoring keeps track of the incoming request types like Market, Product, Amount, No of months, Request origin (CO BRAND) Yahoo, AOL and so on.
Advertisements model hosts ads of various products in their main site and other satellite sites. These ads are chargeable on the cost per click model.

The pain point with the above revenue model here is that a simple functional change could break the entire revenue system (tracking mechanism). This could impact wrong parameters for CPC and Omniture calls getting stored in the DB, tracking of the co brand listings getting missed and the existing call tracing mechanism having browser compatibility issues. So all these had to be manually tracked by QA by running huge set of more than 10000 tests which almost costs 3 – 4 minutes for running each test case across all browsers. This manual effort requires lot of human interaction and concentration and sometimes results in missing the important tests due to tight test schedules.

There are existing Coded UI test developed using Visual Studio Team Test by the team had following severe limitation of:

Supporting only IE browser
No built in mechanism for storing test results

Hence a robust functional automation framework needs to be designed to overcome these challenges which will directly reduce the risk of huge revenue loss. Being a long term partner, they looked at Monocept to solve this humongous problem through right automation framework.

Automation scripts need to run on different browsers like Firefox and Chrome for which a new cost effective tool needs to be evaluated.
The tool also needs to minimize the manual intervention from the QA engineers. Hence the Test Automation exercise must:

Be a seamless automation solution for executing tests across all browsers.
Be easy to execute in batches without requiring manual intervention.
Enable executing the regression suits of > 10000 test cases without slipping the test execution schedule.
Have minimal learning curve and reduce investments in Training
Help in better Resource Management.
Reusing the tested-and-verified functional automation scripts generated in Visual Studio Coded UI for IE will save time rather than reinventing the wheel & developing an automation framework from scratch. Hence the scripts for supporting other browsers need to be generated. These new scripts have to be compatible with Visual Studio IDE to seamlessly integrate and execute all the scripts in one batch.

MONOCEPT’S APPROACH TO THE PROBLEM

Monocept investigated the problem and limitations of Visual Studio of being able to run Automation only in IE and no mechanism for storing test results. It came up with a simple solution to create a customized testing framework to capture and compare the Ad call values from DB with the new results. Selenium was used in Integrated mode with VS to extend running existing tests on other browsers as well. The Reusability advantage, Compatibility with VS IDE and Training needs are all met by generating scripts using Selenium web driver.

Selenium web driver requires installing drivers for each browser separately to be able to generate scripts in C Sharp. This helps in being compatible with executing the scripts on Visual Studio IDE.

Proof of Concept is prepared for automating the initial smoke tests which gained the confidence of the Client.

The POC was extended to automate all > 10000 regression test cases using this approach.

Monocept’s technical expertise in finding simple yet cost-effective solutions for complex problems and experience in core engineering practices gave an edge and confidence to client that Monocept will beat the expectations and deliver the results.

SOLUTION

Monocept has come up with two solutions to the client’s challenges stated above.
The first solution was creating a custom testing framework which

Takes care of capturing all Http requests using fiddler core DLL, filtering Omniture/CPC , Ad calls and writing it to the local xml file and then
Parse the xml file and create respective Omniture/CPC and Ad objects and
Provide methods to get corresponding values from database and
Storing test results into an xml file and uploading it into TFS.

And the second solution was to use the selenium based coded UI testing.

The scripts generated for IE using VS Coded UI test are reused to save automation time. Monocept generated scripts for all browsers using the Selenium IDE and exported the same to Visual Studio IDE for execution.
The scripts generated using VS coded UI test and new scripts generated using Selenium web driver are integrated to run test in batches, without manual intervention.
Selenium web methods are extensively utilized to enhance the automation capabilities like automatic screenshot capture which executing the scripts.
Configuration changes in Selenium web driver scripts helped in overcoming the challenges of extending the support to cover Safari browser too.
With this solution it was easy to integrate test cases with Microsoft test manager and TFS.

PROJECT STATS

Technologies/Tools Used:

i. Visual Studio IDE for Coded UI automation
ii. Team Foundation Server
iii. Selenium web drivers for Chrome and Firefox
iv. Dot Net Framework 4.0
v. Fiddler tool