Why does a test fail when all the tests are run, but passes when run individually?

When many test cases are run together, certain test cases can effect the results of other test cases. It is possible to have a test case the fails when in a group, but passes when run individually.

Many of the CDRouter test cases depend on the WAN or LAN protocols operating correctly. If the router FAILs a test case, the router may be left in a state where the next test can not run without failing.

As an example, if a router fails a DHCP renumbering test on the WAN interface, it may be left in a state where it does not have a valid DHCP lease. If the next test involves forwarding packets from the LAN to the WAN, the test will likely fail. CDRouter does not try to recover the failed protocol before starting the next test.