One aspect of TR-069 firmware download testing that should be taken into consideration involves the download and installation of firmware that differs from the original firmware installed on the DUT. If alternate firmware is installed during a firmware download test, the behavior of the DUT may change, inadvertently producing inconsistent results for all subsequent test cases in the test run.
We recommend two different test strategies for verifying TR-069
firmware download functionality using CDRouter’s OD-128 firmware
strategies are easy to implement with CDRouter and are applicable to
all TR-069 devices that support firmware download functionality.
Strategy 1: Run all firmware download tests using the original version of firmware installed on the DUT
The first strategy is to simply perform the OD-128 firmware download tests as part of a larger test package using the same firmware that is already installed on the device under test.
With this strategy the firmware download mechanism of the DUT is verified while also ensuring consistency within the context of the larger test run. The trade-off with this strategy is that although the download mechanism is verified, the DUT’s ability to upgrade or downgrade to specific alternate versions of firmware is not explicitly tested and verified.
Strategy 2: Run the firmware download tests independently using multiple versions of firmware
A second strategy which addresses the trade-off above is to isolate and independently perform all firmware download tests using multiple alternate versions of firmware.
This strategy involves the creation of multiple CDRouter configuration files. Each configuration file would specify an alternate version of firmware for the firmware download tests, with all other settings being the same. Testing would involve running each of CDRouter’s OD-128 firmware download tests independently while cycling through all of the individual configuration files that have been created. The basic sequence of tests would be as follows:
od128_test_8.1using CDRouter config A
od128_test_8.1using CDRouter config B
od128_test_8.2using CDRouter config A
od128_test_8.2using CDRouter config B
Assuming config A above specifies upgraded firmware and config B specifies the original firmware, the DUT would be upgraded and downgraded to the firmware versions specified in configs A and B, respectively, for each firmware download test. This can be easily expanded to an arbitrary number of firmware versions by simply creating additional CDRouter configuration files that each specify a different version of firmware for the firmware download tests.
The trade-off with this strategy is that the firmware download tests must be isolated and run independently since multiple CDRouter configuration files are required. However, this strategy can be fully automated using either a simple shell script and the CDRouter command line interface, or the test queue feature added to BuddyWeb as part of the CDRouter 5.0 release. This makes it possible to implement the firmware download tests within the scope of a larger test run. For example, the sequence of events for a complete automated test run for a particular device may look something like this:
- Execute Test Package 1 (all base functionality not including TR-069)
- Execute Test Package 2 (TR-069 - OD-128 Tests 1 through 7)
- Execute Test Package 3 (TR-069 firmware download - OD-128 Test 8.1 Config A)
- Execute Test Package 4 (TR-069 firmware download - OD-128 Test 8.1 Config B)
Using CDRouter 5.0’s data management features to group all test results together makes identifying and filtering all data associated with a particular test run very easy as well.