How to choose the right automation framework

Introduction

Automation framework is a set of guidelines / standards / programs / tools defined to improve the productivity of automation resources with better reusability of scripts and ease of use. Framework should also simplify the maintenance effort of the automated tests. If you build / use a strong automation framework, you don't need highly experienced automation resources to build automated tests. Even a junior automation resource with basic understanding on the tool can automate a product if the framework is built to address all the necessary features to automate given product.

Framework should also improve the reporting ability of automated scripts, with better visibility of failures with use of screenshots, logs and, videos in some cases.

The framework should help organizations get better ROI on automation projects.

Can I automate without a framework?

Yes! It is not mandatory to use a framework to implement automation.

However, the automation script will become cumbersome as more and more tests are automated without a strong framework. Maintenance of such scripts becomes huge overhead for the automation teams and may lead to abandonment of total automation initiatives.

Sometimes, automation without frameworks (mostly Record & Playback approach) may work for santiy tests with very few test cases (less than 30). Investing lot of time on frameworks for such smaller initiatives may not be feasible.

Automation frameworks

There are different automation frameworks available / used in the industry for different needs. While most of them are custom built, some of the frameworks for open source tools are are readily available for use with basic features.

Most commonly used frameworks in the industry include Data-driven, Modular, Keyword-driven and Hybrid. Each framework can be built with basic & advanced features.

Data-driven Framework:

• Used when same tests are performed with different data sets
• Tests & Test Data are represented by separate data sets for each feature
• Easy to add/edit tests & test data
• Test data maintained in external data source like excel, databases, flat files etc.

Modular Framework:

• Used for small & stabilized applications with limited independent features
• Each test is represented by separate module / function
• Test data is passed as arguments to functions

Keyword-driven Framework:

• Actions & transactions are represented by simple keywords
• One or more records in Datasheet for each manual test step
• Easy to understand / implement / maintain code

Hybrid Framework:

• Combination of multiple frameworks
• Usually it’s keyword-driven framework with data-driven features
• Common data source is used for different types of tests

There is a lot of buzz in the market for script-less automation tools / frameworks these days. These frameworks allow non-technical manual testers and business users automate their own test/use cases without much understanding on the automation tools, programming and frameworks.

Why so many automation frameworks?

Each disease needs different medicine. Each of these frameworks solves different issues.

Here are the advantages & dis-advantages of different frameworks.

Advantages	Dis-advantages
Data-driven Framework
§  Easy to develop & maintain §  Easy to automate using this framework §  Takes less effort to develop	§  Suitable only for tests with repeated data sets §  Need to build separate script & data sets for each functionality
Modular Framework
§  Not quite complex to build §  Reusability of code to some extend §  Well organized and easy to understand	§  Maintenance effort of scripts is very high §  Difficult to automate large projects using this framework
Keyword-driven Framework
§  High reusability of code §  Flexible and easy to use §  Improves productivity significantly §  Reduces the script maintenance effort	§  Takes lot of time to build §  Need advanced automation engineers to develop the framework
Hybrid Framework
§  Comes with several features together §  Flexible and easy to use §  Improves productivity significantly §  Reduces the script maintenance effort	§  Takes lot of time to build §  Need advanced automation engineers to develop the framework

Which framework do I need?

Success of an automation project starts with the selection of right framework. Here are different framework recommendations based on the application nature, size and complexity.

Application Nature and Recommended Frameworks
Small Application with less than 100 test cases
Simple tests (with 1-5 test steps) & stable application	Record & Playback
Medium size tests (with 5-15 steps)	Modular Framework
Complex tests (beyond 50 steps) with database validations	Modular Framework - with advanced features
Tests with repeated data sets	Data-driven Framework (with combination of Modular based on above conditions)
Mid-size Application with 100 to 500 test cases
Simple tests (with 1-5 test steps)	Modular Framework
Medium size tests (with 5-15 steps)	Modular Framework
Complex tests (beyond 50 steps) with database validations	Keyword-driven Framework - with database capabilities
Tests have repeated data sets	Hybrid Framework with Modular/Keyword-driven & Data-driven combination
Large Application with 500 to 2000 test cases
Simple tests (with 1-5 test steps)	Keyword-driven Framework
Medium size tests (with 5-15 steps)	Keyword-driven Framework
Complex tests (beyond 50 steps) with database validations	Keyword-driven Framework - with database capabilities
Tests have repeated data sets	Hybrid Framework with Keyword-driven & Data-driven combination
Huge Application with above 2000 test cases
Simple / Medium / Complex tests	Advanced Keyword-driven Framework
Tests have repeated data sets	Hybrid Framework with Advanced Keyword-driven & Data-driven combination

By

Automation Mentor

www.automationmentor.in

We provide hands-on training on automation tools and frameworks