What tests need to be automated?

What tests need to be automated?

While everything is practically automatable, it is important to understand what is needed to be automated in order to get more value from the efforts spent on automation. It is also important to understand what tests need to be automated first.

Over 30% of the automation projects in industry end up in trash for different reasons, most common reason being the selection of wrong features / tests for automation.

Frequently executed tests

Everyone accepts that these are the best tests to start automation with. If you have 10 features in your application with 1000 total test cases, we need to sort the features based on their frequency of execution to pick up the right tests for automation. It is a good idea to start calculating the frequency based on the historical data. This data will give us an insight into how many iterations the regression tests were executed for each feature in last one year. It is also important to understand the trend to estimate the frequency for upcoming year.

Feature	Total Regression Executions
	Q1 2013	Q2 2013	Q3 2013	Q4 2013
Feature 1	8	6	6	5
Feature 2	3	4	5	5
Feature 3	6	3	2	5
Feature 4	5	4	5	4
Feature 5	1	2	1	1

In the above table, I probably give high importance to Feature 2. Though the frequency is less compared to Feature 1, the trend shows that the total executions are expected to be more in coming quarters. The execution count for Feature 3 fluctuates a lot indicating that the feature went through unexpected level of changes in last few quarters and it’s difficult to estimate the future regression cycles. I probably give little less importance to Feature 3.

Once we prioritize the features, we can pick up top 2-3 features (based on the resources available) and initiate automation.

Top priority tests first

If the identified features have more than 100 test cases, it takes few person weeks to complete the automation for the current feature before we pick up next feature for automation. In such cases, it would add value to first automate all high priority cases from all the identified features and then automate the medium and low priority cases.

 If a high priority test fails, it will have great impact on the business, while we can still handle the customers if there are multiple UI issues in the applications. It is highly important to ensure that no high priority test fails in the application.

Feature	Total Cases	High Priority	Medium Priority	Low Priority
Feature 1	200	60	100	40
Feature 2	200	50	120	30
Feature 3	200	30	110	60

We recommend automating the high priority tests for Feature 1, 2 & 3 before automating the Medium priority cases from Feature 1.

Tests that take more execution time

For some reason, if we don’t have priority set up for the test cases or we don’t wish to follow this approach, other best way is to prioritize the test cases based on their execution time. The foremost purpose of the automation is to save time and we obviously we want to quickly automate the tests that take very long time to execute manually.

We might have already performed the regression multiple times by the time we decide to automate (except in the case of TDD). We are well aware of the time it takes to execute each test. If it’s not possible (or time consuming) to identify the execution time for each test case, we should come up with effort for each scenario or sub-feature or group of related tests and divide the effort with total cases to come up with test case level execution effort. Once we have the execution time for each test case (or set of cases), we can sort them based on the time it takes and start automation with top 100 test cases. We can pull the next set of 100 cases once the initial cases are automated.

Execution Time Based on Test Cases:

Feature	Test Case	Execution Time (in Minutes)
Feature 1	Test Case 1	15
Feature 1	Test Case 2	10
Feature 1	Test Case 3	5
Feature 1	Test Case 4	25
Feature 1	Test Case 5	18
Feature 1	Test Case …	…
Feature 2	Test Case 1	35
Feature 2	Test Case 2	6
Feature 2	Test Case 3	12
Feature 2	Test Case 4	25
Feature …	Test Case …	…

Execution Time Based on Test Sets:

Feature	Test Set	Total Cases	Total Execution Time (in Minutes)	Execution Time per Case (in Minutes)
Feature 1	Test Set 1	25	450	18
Feature 1	Test Set 2	10	225	23
Feature 1	Test Set 3	30	510	17
Feature 1	Test Set…
Feature 2	Test Set 1	5	40	8
Feature 2	Test Set 2	55	1050	19
Feature 2	Test Set 3	20	330	17
Feature …	Test Case …

Tests that break frequently

In addition to the above, we can also think of automating few additional cases from remaining features that have frequently failed in last few releases / builds. If you are using a good defect tracking tool and have a practice to tag test case id to a defect, this task would not take much time. Tests that are fixed in previous build and reopened multiple times in subsequent builds have more probability to fail again. Such tests needed to be rechecked as many times as possible as the effort keeps growing as the product grows.

By

Automation Mentor

www.automationmentor.in

We provide hands-on training on automation tools and frameworks

Object Identification in Selenium

Object Identification:

Object identification is a very crucial part of any automation effort irrespective of the automation tool we may use like QTP, selenium or any other. For every automation engineer, It is very important to identify the objects used in the automation effort effectively.

Object identification, to be more precise, objects being identified accurately demands certain steps to be followed and is certainly not a straight forward experience in Selenium RC or Selenium WebDriver.

Selenium WebDriver offers a range of locators to capture a specific object in AUT, like – ID, Name, Xpath, CSS, etc. Most of the times, user has to make use of Selenium IDE, Firebug or Firepath to identify the objects. There are few limitations with the traditional approach:

§  User wants to verify text without triggering an event on the application (like Click)

§  Selenium-IDE frequently generates index-based Xpath, which is not the right approach as maintainability becomes an issue

§  Selenium -IDE locators work only on single HTML reference at a time. Whereas web technology allows nested HTML structure with frames. If you want your Selenium WebDriver to act on an object which belongs to frame/iframe, you must select the frame first.

There are some tools such as Firebug and Firepath that can provide a much deeper understanding of the objects. The drawback here is that it demands understanding of HTML DOM to help you decide how to deal with objects to derive the best object identification locator.

Identifying Objects is Selenium (WebDriver) is pretty simple if you are working on an application built on Firefox, but it gets very tricky if you are working on IE application. So it would be best to master the object identification techniques.

If you have used the Selenium IDE for any recording you would have noticed that there are multiple ways of identifying an object. By default the Selenium IDE tries to recognize the object by the ID or name attribute. Selenium IDE identifies the objects by the attributes mentioned in the order below:

§  ID

§  Name

§  Link Text

§  CSS Selector

◦     Based on Combination of HTML tag, id, class, and attributes

◦     Tag and ID (e.g., css=a#test)

◦     Tag and Class (e.g., css=a.test)

• Tag and Attribute (e.g., css=input[name=lastName])

§  Xpath – Attributes

§  Xpath – Relative

§  Xpath – Absolute or Position

§  DOM

◦     getElementById

◦     getElementsByName

◦     dom:name

◦     dom:index

Example:  When we are identifying a search text box, we can locate the object using different locators as shown below.

When we record a simple step of clicking on the “Google Search” button on the home page. If you can convert the recorded test step to Java – JUnit 4 – WebDriver (Options -> Format -> JUnit 4 – WebDriver). The code for clicking the “Google Search” button using ID attribute would look like this -

driver.findElement(By.id("gbqfb")).click();

for name

driver.findElement(By.name("btnG")).click();

for cssSelector

driver.findElement(By.cssSelector("#gbqfb")).click();

for Xpath Attributes

driver.findElement(By.xpath("//button[@id='gbqfb']")).click();

for Xpath Relative

driver.findElement(By.xpath("//div[@id='gbqfbw']/button")).click();

for Xpath Position / Absolute

driver.findElement(By.xpath("//button")).click();

If you prefer not to use Selenium IDE you can use Firebug or Firepath to identify the objects

Firebug:

Example: Identifying a object locator using Firebug

Firepath:

Example: Identifying a object locator using Firepath

Object Maintenance:

How do we manage objects in the repository efficiently so that we can adapt any change in future in a very efficient manner?

Applications go through constant change with the additional of new screens / objects or changes to the existing. We must know which objects are already present in the repository and which objects in AUT have undergone a change and require update in Object Repository. Many are lost in this phase, ending up with relearning the objects / making changes to object without knowing the impacted test cases. As a result you have to maintain duplicate objects or notice unnecessary failures during execution.

As mentioned earlier, the first step to help with Object Maintenance is Object Identification. Smarter object identification helps ensure that you only have to deal with object maintenance, if an object has truly undergone a change.

An automation tool that can help you understand the impact of any object change, like “the list of test cases impacted by an object change” will save lot of maintenance time. Unfortunately, most of the available automation tools identify the change in application only after a failure has occurred during execution!

Impact of Object Management on Automation ROI

Object management that looks beyond the basics of object identification will have a much larger impact on ROI from a test automation exercise. Incomplete and under-utilized Object Management will have adverse consequences such as:

§     Productivity slows down

§     Maintenance efforts increase

§     Overall cost is increased as against anticipated savings

§     Above all, readiness of automation suite is not guaranteed when required

 By

Automation Mentor

www.automationmentor.in

We provide hands-on training on automation tools and frameworks