Strategies for firmware download testing (TR-069)

One aspect of TR-069 firmware download testing that should be taken into consideration involves the download and installation of firmware that differs from the original firmware installed on the DUT. If alternate firmware is installed during a firmware download test, the behavior of the DUT may change, inadvertently producing inconsistent results for all subsequent test cases in the test run.

We recommend two different test strategies for verifying TR-069 firmware download functionality using CDRouter’s OD-128 firmware download tests, od128_test_8.1 through od128_test_8.7. Both strategies are easy to implement with CDRouter and are applicable to all TR-069 devices that support firmware download functionality.

Strategy 1: Run all firmware download tests using the original version of firmware installed on the DUT

The first strategy is to simply perform the OD-128 firmware download tests as part of a larger test package using the same firmware that is already installed on the device under test.

With this strategy the firmware download mechanism of the DUT is verified while also ensuring consistency within the context of the larger test run. The trade-off with this strategy is that although the download mechanism is verified, the DUT’s ability to upgrade or downgrade to specific alternate versions of firmware is not explicitly tested and verified.

Strategy 2: Run the firmware download tests independently using multiple versions of firmware

A second strategy which addresses the trade-off above is to isolate and independently perform all firmware download tests using multiple alternate versions of firmware.

This strategy involves the creation of multiple CDRouter configuration files. Each configuration file would specify an alternate version of firmware for the firmware download tests, with all other settings being the same. Testing would involve running each of CDRouter’s OD-128 firmware download tests independently while cycling through all of the individual configuration files that have been created. The basic sequence of tests would be as follows:

  • Run od128_test_8.1 using CDRouter config A
  • Run od128_test_8.1 using CDRouter config B
  • Run od128_test_8.2 using CDRouter config A
  • Run od128_test_8.2 using CDRouter config B
  • ….

Assuming config A above specifies upgraded firmware and config B specifies the original firmware, the DUT would be upgraded and downgraded to the firmware versions specified in configs A and B, respectively, for each firmware download test. This can be easily expanded to an arbitrary number of firmware versions by simply creating additional CDRouter configuration files that each specify a different version of firmware for the firmware download tests.

The trade-off with this strategy is that the firmware download tests must be isolated and run independently since multiple CDRouter configuration files are required. However, this strategy can be fully automated using a simple shell script and the CDRouter command line interface.