CDRouter Support

Using CDRouter with Jenkins

knowledge-base version 11.1

Using CDRouter with Jenkins

Jenkins is a popular open source continuous integration tool used to manage the workflow of building and testing software. In a CPE firmware build environment, CDRouter can be combined with Jenkins to provide the testing verification of each firmware build.

Jenkins Test Results

Jenkins has built in concepts for testsuites and testcases. It natively recognizes the junit XML test result format. While not all CDRouter test concepts directly map to Jenkins, it is possible to translate CDRouter test results into junit XML test results that Jenkins can understand.

Example Work Flow

Much of the Jenkins process is controlled by scripts that run on the Jenkins server. These scripts handle installing new firmware on target CPE devices and then starting CDRouter to perform testing. This document will not cover the specifics of building and deploying a firmware image since these are vendor specific practices. This document also assumes you are running CDRouter 10.0 or newer.

Using the Web API

CDRouter’s Web API provides full control over executing CDRouter test packages and managing results. QA Cafe has also developed a ruby gem built with the API to make interactions with CDRouter even easier.

In most cases, scripts used to drive CDRouter testing will call out to a CDRouter system to start one or more test packages. The script can wait until the tests are complete and then gather the final results. Scripts may also want to export CDRouter test results for a backup or perform other result processing.

Web API Tokens

When the CDRouter user model is enabled, the web API requires the use of an API token. Each user is assigned an API token automatically. It may be beneficial to create a user specifically for Jenkins automation. Please see the Authorization section of web API for more information.

Using Tags

CDRouter’s tag concept can be used to select test packages and organize test results. Results may be tagged with firmware versions, device versions, or other test environment information.

A Simple Example

This example uses a ruby script and the cdrouter ruby gem. You’ll need to make sure a current version of ruby is installed on your Jenkins server and install the cdrouter ruby gem.

The following ruby script shows how to start multiple packages, wait for completion, and then convert the results into a format Jenkins can understand. Each CDRouter package must be fully automated and run without requiring user interaction.

#!/usr/bin/ruby

# -- Use the CDRouter Gem to make life easy
require 'cdrouter'

sess = CDRouter::Session.new
sess.base_url     = "https://10.0.1.179"
sess.api_token    = "5a21f9c7"
sess.debug        = false


begin
  puts "Connecting to #{sess.base_url}"
    puts "Found CDRouter version " + sess.version
    rescue => exception
      puts "Unable to determine the CDRouter Web API version from #{sess.base_url}"
        puts "Error: " + exception.message
          exit
          end

sess.packages.list( :tagged_with =>"smoketest").each do |p|

    # -- launch this package and tag with "jenkins"
    result = p.launch( :tags => "jenkins,blah", :extra_cli_args => "-testvar myvar=example")

    # -- display a text report
    result.display

    # -- create a jenkins style *.xml test report
    File.write( p.name + "_" + result.name + ".xml", result.to_jenkins )

    # -- write out a csv version
    File.write( p.name + "_" + result.name + ".csv", result.to_csv )

    # -- get the raw logs
    result.logdir_to_tgz

    # -- export a copy of the result for archive
    result.export_to_file

end

This script will produce the following output:

./jenkins.rb
Connecting to https://10.0.1.179
Found CDRouter version 10.0.3
Started CDRouter package BW10 with job id 24
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: running
Checking status of CDRouter job id 24: completed

Test result:

    Summary: The package completed successfully
      Start: 2016-04-22T12:01:21.267301-04:00
   Duration: 150 seconds
    Package: BW10
     Config: PPPoE_perf_v4_2.conf
       Tags: blah,blue,smoketest,jenkins,yellow
      Tests: 37
       Pass: 29
       Fail: 2

 Report URL: https://10.0.1.179/results/20160422120121
  Print URL: https://10.0.1.179/results/20160422120121/print/

Started CDRouter package BW11 with job id 25
Checking status of CDRouter job id 25: running
Checking status of CDRouter job id 25: running
Checking status of CDRouter job id 25: running
Checking status of CDRouter job id 25: running
Checking status of CDRouter job id 25: running
Checking status of CDRouter job id 25: completed

Test result:

    Summary: Fatal error while running start: no-lan-pppoe-address
      Start: 2016-04-22T12:04:06.704265-04:00
   Duration: 87 seconds
    Package: BW11
     Config: PPPoE_perf_v4_4.conf
       Tags: blah,smoketest,jenkins
      Tests: 53
       Pass: 0
       Fail: 0

 Report URL: https://10.0.1.179/results/20160422120406
  Print URL: https://10.0.1.179/results/20160422120406/print/

Additional Notes

The script runs multiple CDRouter test packages. This is useful to verify a CPE device in multiple configurations. Each CDRouter test package is translated into its own unique Jenkins XML result file. Jenkins can handle multiple XML files each with its own set of unique results.

The script also shows examples of exporting the results, generating a csv file of the result, and downloading all the raw log files. These methods can be used for additional result integration.

The cdrouter ruby gem knows how to translate test results into the junit XML format needed by Jenkins. You can see the details of how this translation is done by looking directly at the cdrouter ruby gem: https://github.com/qacafe/cdrouter_ruby_gem

Help Us Improve

The cdrouter ruby gem is hosted on github. We welcome push requests and feature suggestions to allow CDRouter to work better with Jenkins and other CI tools.

Contents

×

About CDRouter

CDRouter is made by QA Cafe, a technology company based in Portsmouth, NH.

Get in touch via our Contact page or by following us on your favorite service: