Skip to Main Content

Top Five Things to Consider in a Test Automation Framework Twitter Facebook Print E-mail
Written by Bryan Robinson   
Tuesday, 25 May 2010

Software quality assurance has become more and more prevalent over the last several years. The demand to produce high-quality, reliable software is ever-increasing, and developers and quality assurance engineers strive to fulfill this demand. In doing so, one cannot overlook the need for a well-organized test automation framework.

A test automation framework has various benefits for the project. It can allow the testers to spend their time testing new development, rather than regressing the system to find where new code has introduced bugs in existing code. Manually testing the entire application would simply take too long and is prone to errors.

In my opinion, several points can be considered in a test framework. Here are the top five on my list and how our team uses each item.

  1. Test Script modularity: This concept encourages testers to write code in an object-oriented manner. This allows reusable code to be incorporated throughout any given test. It can also assist in set-up tasks, such as preparing an environment or creating or refreshing database tables.

    We use WatiN, NUnit and CruiseControl.NET. We have created a useful wrapper for WatiN (which we call WatiNQA). This wrapper gives us added XPath support that is not native to WatiN. C# gives us our test modularity. The WatiNQA wrapper is separated from the actual test framework to maintain the modularity of the frameworks in use.

  2. Data-driven or keyword-driven testing: You may have specific scenarios that require an object in a specific state. As a good measure of verification, you can use Microsoft Excel spreadsheets or database queries to gather information to be tested. You can also leverage this to include database verification after a task has been completed.

    We query the database to find specific data scenarios. This removes the process of setting up each data scenario through the user interface. After the test is complete, we verify that it is being saved correctly in the database.

  3. Environment to support the test framework: This would include a continuous build system to launch a test suite at will, on schedule, or even after another event has occurred. With this environment, you can also run different suites of tests, such as smoke tests, functional tests, or browser-based tests. The ability to review test suite results and to view log files are necessary to provide feedback on the success or failure of the test suite. This helps ensure the tests are contributing to the testing efforts. It is also convenient to have a method of linking automated tests with their corresponding manual test cases and any bugs reported on each feature.

    Using CruiseControl.NET with NUnit allows us to maintain the test framework with ease. We have it configured to launch several different configurations of test suites, and viewing the results is easy. We’ve also added the convenience of posting the automated test results directly into the test case management tool by creating a NUnit add-in. We use additional test decorators: <code>[Test, TestCaseKey(“”), BugKey(“”)]</code>.

  4. Maintenance friendly: The framework offers an appropriate solution for the project at hand. It must be easy to use and maintain. By keeping clean code in an organized fashion, it becomes more of a tool than a hindrance. Documentation is always helpful when using the framework.

    We include comments throughout the tests and supporting class files. The objects are configured to leverage reusable code to make writing a test a simple process. We also implement the Façade pattern for our helper methods. This helps us to simplify the tests but still have access to lower-level functions as needed.

  5. The right tools: Even though many tools exist for automation use, not every tool is adequate for each project. A stable tool that has longevity can provide a standard of excellence on the project. When this happens, the framework is used more and used correctly. Also, the test results have a higher level of trust. This will help avoid automation frameworks to be discarded when changing personnel on the team.

    We use C# and WatiN for several reasons. C# is a stable tool that has longevity and great documentation. WatiN is open source and has an active community working to improve it. Though it was originally created for Internet Explorer, a version is available which supports FireFox testing. WatiN deals directly with the Document Object Model (DOM) and can truly represent how the user would truly interact with the Web site.

Test automation is a full-time job. Something can always be automated to contribute to the project’s progress. It will take time to set up, but once it is established, it can add great value to the project.

Bryan Robinson is a QA engineer for the Church.

Add Comment



Learn how to become a full time or part time Missionary.