CDRouter Developers Guide

Disclaimer

Users may develop and add custom test modules and test cases to CDRouter. Limited technical support is available for the development, verification, and debugging of custom test cases. The QA Cafe Support team will not write or debug custom test cases; only high level guidance will be provided.

All CDRouter test cases are written in Tcl. The underlying protocol library is also largely implemented in Tcl. There is no officially documented test case API, however, the source code for all test cases and most of the underlying protocol library is provided and may be inspected.

There are two primary methods for users to develop and add custom test cases to a CDRouter system:

  • Using the included command line utility for creating a copy of an existing test module or test case; this is the recommended method and is best for users that want to make small changes to existing test cases or create new test cases that are very similar to existing test cases.

  • Creation of completely new test cases; this option is recommended for experienced Tcl software developers only and will require manual inspection and interpretation of CDRouter’s test case source code and the source code for the underlying protocol library.

The source code for all CDRouter test cases and the underlying protocol library are subject to change at any time. The QA Cafe Support team will do its best to alert users of any major breaking changes via statements in the release notes. QA Cafe cannot guarantee backwards compatibility for non-production enhancements to CDRouter, including custom test cases.

Overview

This guide covers the basic concepts involved in modifying QA Cafe’s existing test cases. It is expected that test developers will look at the Tcl source code included with CDRouter under /usr/cdrouter/tests for additional coding details.

Requirements

In order to modify existing test cases, CDRouter must be installed on the system where the tests will be executed. In addition, since CDRouter test development is done using the Tcl scripting language, the reader must have some basic knowledge of Tcl programming.

Test Suites

A test suite is a collection of test cases which is loaded as a unit into the underlying CDRouter software. The directory in which a test suite is stored is known as its testpath. Each test suite has three levels of organization. The top level is the test suite level. The second level is the test module level. The third and last level is the test case level.

“Test organization.png”

The standard test suite is located at '/usr/cdrouter/tests' and is loaded by default when a CDRouter test package is launched.

Test modules are Tcl files. A single test module file contains individual test cases and related Tcl code used by those test cases. Besides test modules, a test suite contains two special files. A file named modules is known as a test suite’s modules file and is used to identify all of the test modules (files) that are part of the test suite. A file named config.dic is known as a test suite’s configuration dictionary and is used to define configuration parameters known as testvars which can be used to control the behavior of test cases. The underlying CDRouter software looks for these two files and automatically loads them if found.

Pktsrc library

CDRouter’s test suite is supported by a protocol engine called pktsrc (“packet source”). This provides the routines and data structures to create all of CDRouter’s emulated hosts and protocol services. During testing, test scripts will make calls to various pktsrc TCL procs in order to send traffic on the network and process information that has been received. The source code for these procs can found in the "/usr/cdrouter/lib/pktsrc<version>" directory, where “<version>” is the version of CDRouter currently installed on your system.

Custom Test Suites

When getting started with custom test suites, we recommend that you find existing tests from the standard test suite that perform similar functionality, and copy them into Tcl module files in your custom test suite. There, you can modify and tailor them to provide the functionality you need.

The files in the standard test suite should never be modified, as this could cause confusion or unintended errors in CDRouter’s operation, and any changes you make will be overwritten whenever CDRouter is upgraded or reinstalled.

Whether you wish to copy an existing test case or develop new tests to add to CDRouter, you can do this by creating a custom test suite in a separate location (eg. '/usr/cdrouter-data/custom') with a layout similar to the standard test suite described above.

Note, it is not necessary to copy procs or any other code from the pktsrc library that might be used within custom test cases.

The 'cdrouter-cli -new-testpath' command automates much of this process by allowing the user to specify a comma-delimited list of existing tests, modules and expansions to copy into a new testpath. The names of the copied tests/modules will be renamed to start with the name of the testpath directory. This ensures the copied tests/modules will not conflict when loaded alongside the originals.

The example below uses the 'cdrouter-cli -new-testpath' command to initialize a new testpath containing copies of the ftp_30 test, nat.tcl module and basic.tcl module.

# -- create a new directory
$ mkdir mycustomtests

# -- move to the new directory
$ cd mycustomtests

# -- create a new testpath
$ cdrouter-cli -new-testpath nat.tcl,basic.tcl,ftp_30
New testpath at mycustomtests created successfully.
...

# -- directory contains copies of specified tests,
# prefixed with mycustomtests
$ ls -l
-rwxrwxrwx 1 root qacafe    398 Jan 16 14:44 config.dic
-rwxrwxrwx 1 root qacafe    202 Jan 16 14:44 errors.dic
-rwxrwxrwx 1 root qacafe   6599 Jan 16 14:44 labels.dic
-rwxrwxrwx 1 root qacafe   2053 Jan 16 14:44 modules
-rwxrwxrwx 1 root qacafe   8366 Jan 16 14:44 mycustomtests_basic.tcl
-rwxrwxrwx 1 root qacafe   1849 Jan 16 14:44 mycustomtests_ftp.tcl
-rwxrwxrwx 1 root qacafe 118757 Jan 16 14:44 mycustomtests_nat.tcl

Once the new test suite has been created, the copied Tcl modules can be opened using a text editor and modified to suit your needs.

The new testpath will also contain prepopulated modules and labels.dic files configured to allow CDRouter to load each module and skip tests or modules when appropriate.

The config.dic and errors.dic files are optional templates that can be customised to support new testvars and error messages. They are not required in order to run your custom test scripts. See the Creating New Testvars section below for more details.

Adding Custom Testpaths

Before the modified tests can be used within the CDRouter web interface, CDRouter must be told to load the new test suite. On your CDRouter system, navigate to the System>Preferences page, add the new testpath to the Custom Testpaths list and press the Save button. A green success banner will appear after a few seconds to indicate that the custom testpath has been loaded and the new tests are ready to use.

“Custom Testpath.png”

After your custom testpath has been added to CDRouter, your custom test suite will be available in the Package editor under Custom in the Expansions column.

“Custom Tests Package.png”

Creating New Testvars

Testvars are parameters used to allow a test suite to be configured at run time. A CDRouter configuration file is a set of testvars set to certain values. Testvar values can be accessed by the test suite during test execution. Testvars can be used to provide configuration, initial values, parameter ranges, etc. The use of testvars in any new test cases is strongly encouraged.

Testvars can be assigned a value in a CDRouter configuration file by calling testvar:

testvar myTestConfigParam 120

Inside a testcase, the buddy::getvar procedure can be used to get the value of a testvar:

set var [ buddy::getvar myTestConfigParam ]

in the above example, the variable var will be assigned the value of myTestConfigParam.

New testvars can be defined in a test suite by adding calls to buddy::testvar_define to the test suite’s configuration dictionary file config.dic. Calls to buddy::testvar_define should have the following form:

buddy::testvar_define myTestConfigParam {
    type { custom }
    default "120"
    description {

This text describes the testvar myTestConfigParam.

    }
}

where myTestConfigParam is the name of the new testvar. The value of the default field is used as the testvar’s value if not set in the configuration file. The value of description field is shown in CDRouter’s testvar documentation browser. Testvars can be used anywhere during the test suite setup or during the execution of a test run:

set var [ buddy::getvar myTestConfigParam ]

if { $value == "120" } {
    buddy::loginfo "myTestConfigParam is set to its default value"
} else {
    buddy::loginfo "myTestConfigParam has been set to $value in the configuration file"
}

See Testvar Properties and Testvar Constraints for more on defining testvars.