Test Cases

Test cases are the most important entity in TestLog. Test cases contain the bulk of the test data and are also used to update the results of the tests as they are carried out. Test cases are housed in Test Suites (to learn more about test suites, see Chapter 7 Projects and Test Suites). One of the key concepts to bear in mind when creating projects and test cases in TestLog is that test cases exist in two flavors.

A Generic Test Case is a test case, which is not yet assigned to a project (to learn more about projects, see Chapter 7 Projects and Test Suites). Generic test cases may be assigned to any or all projects within the database, in which they are created. Once a generic test case is assigned to a project, it exists within that project as a Project Test Case. Any changes or updates made to a project test case only change the test case within the context of its project. Generic test cases do not contain any data concerning the results of their tests because these may vary from project to project.

Example:

I create a generic test case called INST_001, which consists of a basic install test.

I have three projects representing three different versions of my software product. Call them VER_1.1, VER1.2 and VER2.0. The basic install test for each version is the same.

I drag the generic test case INST_001 into each of the three projects. I now still have the generic test case INST_001, but also have three project test cases called INST_001 – one in each project, representing the install test for that version of the software. I can assign testers to each of the project test cases and carry out the tests, after which I update each of the project test cases with the results.

It is also possible to directly create project test cases, but it is recommended that the test cases be created firstly as generic and then added to the projects as needed.

To create a new test case:

Click the ‘Create test case’ button from the main window

Right click a test suite folder in the tree view window and select ‘New test case’.

Click the ‘Create’ menu and select ‘Test case’.

Lets take a look at the differences between the new/edit generic test case dialog and the new/edit project test case dialog

Generic test case dialog

Note this dialog is just one page and does not contain any fields that are concerned with the status or results of the test. Now lets examine the project test case dialog, which is a tabbed dialog with three tabs.

Project test case dialog Pg.1

Project test case dialog Pg.2

Project test case dialog Pg.3

Many of the fields are the same, but you will notice that the project test case dialog contains some extra fields in the ‘Test execution status’ section, which allows the test case to be updated as the tests are carried out. It also has a “Test History” section which displays various history events for the test case, such as creation and any change in execution status.

The button will open a HTML edit application so text can be formatted using HTML elements such as lists, tables and hyperlinks.

Test Case Dialog

Both dialogs contain the following fields.

Test ID

This is an ID, which represents the test case. Test IDs must be unique within each test suite.

Expected Duration

This is a value in HH:MM format, which is how long the test should take to execute. This value is used in reporting to estimate the current rate of progress.

Title

The title, or name of the report. The title should be a descriptive name which allows the user to see at a glance the nature of the test.

Test Type

We have defined the following basic test types.

Functionality tests check the basic mechanics of the software. Many of your test cases will have type functionality.

Usability or Fitness for use tests are more concerned with the user friendliness of the software. Is the application easy to navigate and intuitive. Note: Some organizations use one the term Usability, some use Fitness for use. We have included both for compatibility.

Performance tests put the application under its paces under some kind of load or stress condition. For instance, a text editor opening a very large file, or a distributed application under heavy network traffic may be types of Performance tests. Performance tests generally check the speed and efficiency of the application under these conditions.

Installability tests are concerned with the install and uninstall procedures of the application. Reinstalls and upgrades may fall under this category.

Reliability tests are similar to Performance tests in that they test the application under stress conditions. However reliability tests are more concerned with how stable the application is over a longer length of time under these conditions.

Maintainability and Service tests how easy it is to keep the software up to date. This may include the testing of a Live Update or patching system. Note: Some organizations use one the term Maintainability, some use Service. We have included both for compatibility.

Documentation tests the correctness of the applications documentation and help. Such factors as spelling, grammar, inline linking and indices fall under this category. You may also wish to include any testing of online help under this test type.

Other tests are tests that don’t fall into any of the above test types.

Test Phase

The test phase represents the point in the product life cycle the test is taking place. TestLog has been designed with large-scale software products in mind, and as such we have attempted to incorporate some standard industry test phases.

Unit tests check a particular area of functionality of one component of the software product. This may often correspond to one subroutine in the actual code of the software. Unit tests are usually carried out by developers.

Component tests are concerned with individual components of the software product. This may be one executable or dynamic link library in a product, which will contain many executable binaries.

Integration tests take place once two or more components, which have been designed to work together, are actually put through their paces. Integration tests are often concerned with threading models and inter-process communication.

System tests are concerned with testing the entire software product in a simulated real world environment. System tests will usually take place in the lab on alpha or beta level software.

Acceptance test In comparison to most test phases, which look for bugs and problems, acceptance testing generally tests that the system under test meets its requirements and does what it’s actually supposed to do.

Pilot tests demonstrate that the system under test can perform correctly in a real world environment. This may include ‘soak’ tests where the product is deployed to a few selected customers.

Several means that the test falls under several of the above test phases.

Other test phases are assigned to tests, which do not fall under any of the above test phases.

Priority

A priority chosen from of one of 5 values, from “Lowest” to “Highest”, to assign a priority for this test case.

Created

The time and date the test case was created.

Last Update

The time and date the test was last updated or edited.

Created by

The windows username of the person who created the test case.

Last Update by

The windows username of the person who last updated or edited the test case.

Recommended config. IDs

The test configurations under which the test must be carried out. Click the <<Add button to display a list of configs to choose from to add to this field. The "," character is used as a separator between multiple configs displayed in the field, for this reason avoid using "," in the name of configs. For more on test configurations see Chapter 4 Test Configurations.

Resource IDs required

The test resources required for the test. Click the <<Add button to display a list of resources to choose from to add to this field. The "," character is used as a separator between multiple resources displayed in the field, for this reason avoid using "," in the name of resources. For more on test resources see Chapter 5 Resources.

Requirements

Links for requirements mapping can be added here, to external documents or requirement IDs. The view button will attempt to open the link using the program associated with the file type, for example if you link to a Word document and click view it will open Word and display the document. If you enter an ID instead of a path and have set the Requirement path in the general options TestLog will open the provided program or web service and pass the ID as a parameter. "%TESTCASE_PATH%" can be entered in to the field and when the test case is viewed it will be replaced with the full path to the test case.

Prerequisites and initial condition

Any steps, which must be carried out before executing the test. For instance a certain test file loaded, or a certain version of the software already installed.

Test description and steps

This is the actual test itself. Include all steps required to complete the test.

Expected results

What the tester should observe during and after he completes the steps outlined in the Test description and steps field.

Notes Field 1 & 2

Two extra fields for miscellaneous information regarding the test, these can be renamed using the “Customize notes” tab in the “Customize fields” dialog under the Edit menu

External description link

This allows the creator of the test case to link to or browse for an external document. For instance, it may be useful to include the feature specification or some design documentations for the purposes of understanding the test better.