What's Your Software Index?

Posted in Functional Testing | April 13, 2010 Roundabout

What is your software index? Most companies intend to get the most value for their dollar when purchasing or developing software. In the testing world there are many avenues that can lead you to a dead–end, or worse - lead you to end up like Chevy Chase's character in "National Lampoons European Vacation": on a never ending round-about. Is there a way to map a road to success using a testing tool? Whether you are a developer planning a software application or a QA manager planning how to test that application, planning is the key, and mapping out a route that avoids detours and unnecessary stops will save time and money.

The first road to map is the requirements. What are the software requirement specifications? What is the functionality of this application or web site? A complete description of the behavior of the system is necessary to make sure all requirements are covered. This would include a set of use cases that describe all the interactions the users will have with the software. In addition to use cases, the SRS also contains non-functional requirements.

Non-functional requirements are requirements which impose constraints on the design or implementation (such as performance engineering requirements, quality standards, or design constraints). (Shelly, Rosenblatt)

Documentation that is clear and concise and include details of each step and validation requirement of each test case will help determine if your testing software will be able to cover each requirement.

The next road is efficiency. Record and playback most of the time is not re-usable when new builds are lurking at the next crosswalk. If the developer's code changes, then in order to have a complete and accurate test you will have to re-record the test. If your tests are developed carefully using objects and methods that are unlikely to change against objects, and the developer's code changes, you can still run the test making a few adjustments. This is important for regression testing on new builds. What if the items you want to validate are dependent on factors that are sometimes enabled and sometimes not enabled, perhaps you have two pages that are alternately used? To truly test user interface, which incorporates all kinds of users from experienced to novice, you must have the ability to choose where and when your validations will come in to play. The ability to include comparison statements where you need them makes all the difference in the world.

The next road is value. Compare what you bought versus what you use. What value are you getting for your expenditure? If your testing tool is being used by a developer, chances are he is knowledgeable enough to be able to write complex scripts that will incorporate code into record and playback scripts and that would be sufficient for covering all requirements. However this takes time away from development and trained testers are more likely to test all aspects of an application. However if you have a testing team that have the knowledge and skill to test efficiently, but do not have the knowledge to write complex scripts, then you are still dependent on the developer, and your testing tool is not allowing the two teams, development and testing, to work independently of each other. It is a complex merge of two departments that utilizes developers and QA. If a testing tool is complicated and the testers are only using 1 out of 50 modules (ex. record and playback), then you are only using about 5% of the tool's value. That is a waste of 95% of your investment.

How many years will you use the software? Let's say 5 years multiplied by the cost for software not used: 5 * $1642.00 = $8210.00. That is what is depleted from your budget that could be put to better use. What if you need 5 licenses? You can see where the money lost quickly adds up.

There are many automated testing tools in use today, and designing an automated test has become an art in itself. Automated test development requires experience, discipline, and training to maximize the benefit of investing in a testing tool. There are many avenues that connect the roads you have already taken.

Who will develop the requirements, stories, and write the automated tests?:

  • Trained automated testers to sign off on requirements
  • Untrained testers can record and playback (5% Rule)
  • Trained testers can utilize the tool at 100%

Who will train employees to use the testing tool?:

  • Training all testers is expensive
  • Outsourcing to an automated testing facility saves on payroll
  • Outsourcing to trained testers also insures that the tool is used at 100%
  • Train a QA Manager, who can then train others

Who is your contact for ongoing support?:

  • Support technician
  • Rely on in-house developers
  • Trained QA Managers
  • Outsource to a trained QA team

In summary, knowledge of requirements, efficient use of testing tools, and outsourcing to a trained QA team versus training in-house are choices that a Project Manager, Developer and QA Manager must consider. Acquiring a tool that fits your testing needs and using the tool to its full potential are all decisions that can put you on the map, and keep your project straight on the road to success.

Lisa Corkren is a certified technical lead for DeRisk IT Inc. She specializes in automated testing and project management with a strong amount of experience with AutomatedQA's TestComplete. She has worked on numerous platforms of both Manual and Automated Testing.