Automation Testing approach - Which is best?

Question

Which is the best suited automation approach? is it Data driven testing or Keyword driven testing? Now even we have Business process testing, what according to you is best approach?

Answer 1

It depends what you are trying to accomplish in your particular situation. One thing that is universal, however, is that test automation is software development, and needs to be treated as such. Use sound design and coding practices. It sounds like you are new to automation. Often a good place to start is building a smoke test that will quickly run for each new build.

Answer 2

It sounds like your question is specific to HP's QTP/BPT. Here are the differences. QTP is really the 'engine' that is used by both BPT and QTP. QTP provides two main views for working with scripts.

QTP: The first is the 'Keyword' view which presents a tree of actions and functions. This mode is suited for beginners and allows them to select objects from either the application or from the object repository as well as selecting the methods to be performed on the objects. The script is then generated automatically.

The second is the 'Expert' view which allows the display and editing of the scripts source code. The expert veiw provides an IDE for developing the scripts. This view is suited for advanced users.

BPT is based on the concept of creating test cases from re-usable business components which in turn are created from keywords. This keyword-driven approach simplifies the process of test case creation by providing a gui that allows the user to 'drag and drop' components to build the test. the script is generated automatically.

Answer 3

It all depends on your needs. As a long-term investment I recommend keyword-driven or hybrid keyword/data driven framework.

See my short overview below (taken from my blog page http://automation-beyond.com/category/automation/automation-methodology/practices/).

Front-End Test Automation Practices – Record/Playback

1. Description • Hard-coded data • Hard-coded flow • No error handling • No or limited reporting • No structure • No verification • No validation

2. Advantages • Easy to create • Quick to create • No programming required

3. Disadvantages • Very low usability • Extremely high maintenance cost • No-proof test results • Break on any fail • Poor test flow coverage • A lot of manual work is still required

4. Applicability • Demo and sale presentations • Same day testing (re-record from scratch, short test flow) • Exploratory testing (investigating how the Testing Tool handles the Application) • Automated data entry (restrictions apply)

Front-End Test Automation Practices – Record/Playback Enhanced

1. Description • Parameterized data • Dataset is a part of the script but not in the code • Verification via GUI checkpoints (hard-coded) • Hard-coded flow • No error handling • No or limited reporting • No structure

2. Advantages • Easy to create • Quick to create • No programming required, but expertise on the Testing Tool is a must

3. Disadvantages • Vulnerable scripts • No multi-environment support • Very high maintenance cost for checkpoints • Not reproducible test results • Break on any fail • Limited test flow coverage • All analysis and validation done manually

4. Applicability • Single/stable environment • Short flow Test Cases • Limited set of checkpoints, as any Database / Data Input change breaks verification and requires recapture • Short-term simplified automation goals

Front-End Test Automation Practices – Data-Driven Framework

1. Description • Programmatically created • Parameterized, capable to import spreadsheets • GUI/Database checkpoints, hard-coded and/or parameterized • Library-based structure • Possibly error handling • Hard-coded yet data-driven flow (input and logic) • Standard reporting • Verification is limited to Testing Tool’s capabilities • No validation

2. Advantages • Good usability and re-usability • Good test flow coverage • Multi-environment support • Data and code are separate • Reproducible test results

3. Disadvantages • Quality and coverage strongly depends on the automation skills of implementing person • Continuous maintenance issues due to a big amount of the code • Exit on fail • Manual validation is required

4. Applicability • Good fit for a single application testing, featuring multi-environment, big datasets, and rarely changing Test Cases • Batch-run capable with additional development efforts • Limited set of checkpoints, as any Database / Data Input change breaks verification and requires recapture • Shared across the team workspace (environment, data, etc.) required (possible issues with offshore support) • Mid-term automation goals

Front-End Test Automation Practices – Keyword-Driven Framework

1. Description • Pure programmatically • Parameterized, capable to import spreadsheets • GUI/Database checkpoints, hard-coded and/or parameterized • Framework-based structure • Limited error handling • Keyword-based flow (logic and data in a spreadsheet) • Possibly extended reporting • Verification is limited to Testing Tool’s capabilities • No validation

2. Advantages • Good usability and re-usability • Compact code • Test development doesn’t require programming skills • Good test flow coverage • Multi-environment support • Data and code are separate • Reproducible test results

3. Disadvantages • Requires initial investment for design and implementation of the Framework • Due to keywords limit doesn’t allow covering very complicated Test Cases • Meta-language specific training is required for staff • Version control issues • Exit on fail • Manual validation is required

4. Applicability • Good fit for a multi-application testing (same platform), featuring multi-environment, big datasets, and a big number of short and straight Test Cases • Test Plan / Test Scenario execution (Batch run) • Limited set of checkpoints, as any Database / Data Input change breaks verification and requires recapture • Better support of distributed teams, especially with extended reporting implemented • Multi-tool support for the meta-language • Mid- and long-term automation goals

Front-End Test Automation Practices – Hybrid Keyword/Data Driven Framework

1. Description • Pure programmatically • Internal Data Model, capable to import/export data from various sources • GUI/Database checkpoints, parameterized/transitioned • Business Verification Rules • Framework-based structure; capable to integrate external objects (i.e. MSXML DOM) • Exception handling and recovery capabilities • Test Case oriented execution (business logic outside of the code) • Data-driven input & verification • Extended business reporting • AI-based validation methods

2. Advantages • High usability and re-usability • Compact and scalable architecture • Test Plan / Test Scenario creation doesn’t require programming or training on the Testing Tool • Extensive test flow coverage • Multi-application support • Data and code are separate • Reproducible and proven test results, convenient and transferable Test Reports • Built-in Consistency and Severity Validation

3. Disadvantages • Requires initial investment for design and implementation of the Framework

4. Applicability • Comprehensive Functional Testing for multi-application, multi-platform products, featuring huge number of complicated Test Cases • Smoke-Regression-Sanity Testing Cycle with extensive coverage and validation • Test Plan / Test Scenario execution (Batch run) • Big and changeable data set and data transitioning cases • Excellent support of distributed teams • Integration with other testing tools • Mid- and long-term automation goals