CDRouter Storage User Guide

Introduction

CDRouter Storage is an expansion for CDRouter, the industry’s leading CPE and router test platform. CDRouter Storage adds network attached storage (NAS) test capabilities to CDRouter.

In a residential or SOHO environment, NAS functionality is typically integrated directly within the CPE device or included in a dedicated standalone device. CDRouter Storage supports both integrated and standalone NAS implementations over a wide range of IPv4 and IPv6 network configurations. CDRouter Storage provides an instant regression environment for testing NAS functionality, and is powerful, easy to use, and fully automated.

CPE Devices with Integrated NAS Functionality

Modern CPE devices can support NAS functionality directly by either including an internal hard drive, or providing one or more USB ports for attaching external hard drives. CPE devices with this type of integrated NAS implementation typically include management tools for provisioning access to the attached storage devices from clients on the LAN and WAN. In many CPE devices LAN access is allowed while WAN access is restricted. However, some implementations do provide external access to the attached storage devices from the WAN.

The test setup for a typical CPE device with integrated NAS functionality is shown below:

NAS Devices Connected to LAN Side of CPE

CDRouter Storage also supports the testing of standalone NAS devices connected to the LAN side of a CPE or gateway. Standalone NAS devices focus solely on providing NAS related functionality, and as a result are often more powerful and feature rich than their integrated counterparts.

Standalone NAS devices are usually only accessible to clients on the LAN, although some do support access from the WAN using a few different techniques:

• UPnP: Some standalone NAS devices utilize UPnP to automatically modify the CPE’s firewall or NAT configuration to allow access from the WAN. This technique relies on the CPE supporting UPnP, and also assumes that the UPnP implementation of the NAS device is interoperable with that of the CPE.

• Third party software: Some standalone NAS devices utilize third party cloud- based applications or services to allow access from the WAN. These applications require active WAN connections and may limit access on the LAN if the CPE’s WAN connection is down or inoperable.

Another alternative for providing WAN connectivity to any NAS device on the LAN is to configure one or more virtual services on the CPE. This requires some additional knowledge of the protocols and ports involved, however many CPE devices include virtual services templates for common protocols which simplifies the configuration process.

The test setup for a typical standalone NAS connected to the LAN side of a CPE is shown below:

File Sharing Protocols

CDRouter Storage includes a number of test modules designed to verify the most common network file sharing protocols found in today’s leading CPE and SOHO NAS devices. CDRouter Storage allows the following protocols to be tested over the LAN:

• SMBv1/CIFS: Server Message Block / Common Internet File System
• FTP: File Transfer Protocol
• FTPS: FTP Secure

Likewise, the following protocols can be tested over the WAN:

• FTP: File Transfer Protocol
• FTPS: FTP Secure

Separate test modules for both IPv4 and IPv6 access are provided for each of the above protocols. This level of granularity allows issues or inconsistencies within a NAS implementation to be quickly identified and isolated. Note that the initial release of CDRouter Storage supports SMBv1 only. Support for additional storage protocols may be included in future releases.

Test Methodology

Initial Setup

CDRouter Storage can be used to test both integrated and standalone NAS devices, as discussed in the Introduction. For most CPE or gateway devices with integrated NAS functionality, the physical storage component is a hard drive or flash drive attached via USB, although some CPE devices have internal hard drives as well.

In a standalone NAS device the storage component is usually an internal hard drive. However, some standalone NAS devices provide USB ports for attaching USB storage devices, and still others include both an internal hard drive and USB ports for expansion. Note that when testing a standalone NAS, a CPE or gateway device must also be included in the test setup.

CDRouter Storage can be used with any of the device configurations described above, provided the physical storage component is properly formatted and recognized by the CPE or NAS device prior to testing. The disk file system used on the physical storage device is irrelevant, although a good testing exercise is to verify that all storage related functionality is consistent across the various file systems supported (FAT, NTFS, EXT, HFS, etc.) by the device under test (DUT).

Some NAS implementations require the installation of dedicated application software on all clients prior to use. In addition, some implementations also utilize third-party cloud-based software to bypass the CPE device’s firewall and allow access to the NAS from clients on the WAN. CDRouter Storage is not compatible with these proprietary solutions and assumes that the DUT has native support for the NAS protocols being tested.

Start-Up

CDRouter Storage does not modify or impact the normal CDRouter start-up procedure performed at the beginning of every test run. However, there are two procedures that occur at the start of each storage related test. First, the SMB or FTP user configuration to be tested is determined, and second, the IP address of the server for the chosen user configuration is resolved. Both procedures are discussed in more detail below.

SMB and FTP Users

CDRouter Storage allows up to 10 SMB and 10 FTP user configurations to be created. Each SMB and FTP user configuration is defined as a unique smbuser or ftpuser testvar_group, respectively, in the CDRouter configuration file. A full description of all of the testvars available for each smbuser or ftpuser testvar_group instance is defined in the Configuration section of this User’s Guide.

At the start of each storage test CDRouter will choose an smbuser or ftpuser configuration to test. CDRouter chooses the smbuser or ftpuser configuration to test sequentially; for the first storage test in the test run, CDRouter will select the first defined smbuser or ftpuser configuration. For the second test CDRouter chooses the second defined smbuser or ftpuser configuration, and so on.

To run any SMB related test cases, at least one smbuser testvar_group must be included in the CDRouter configuration file. Likewise, to run any FTP or FTPS related testcases, at least one ftpuser testvar_group must be defined. If only a single smbuser or ftpuser configuration is defined, that configuration will be used for all storage tests that are performed.

Some test cases require either ReadWrite or ReadOnly permissions. CDRouter will automatically search the list of defined smbuser or ftpuser configurations for a user with the appropriate permissions for each test that is performed.

Server Address Resolution

At the start of each storage related test, CDRouter must resolve the IPv4 or IPv6 address of the DUT, based on the configured SMB or FTP server name for the smbuser or ftpuser configuration being tested. Address resolution is handled a few different ways based on the specific test that is being executed. See the four sections below for a description of how CDRouter handles DUT address resolution based on LAN or WAN access over IPv4 and IPv6.

Alternatively, a user may choose to configure an explicit IPv4 or IPv6 address for the server instead of a name. In these cases CDRouter will not perform address resolution and will instead use the specified address as the endpoint of the test.

Once the address of the DUT is identified, CDRouter will create an SMB or FTP client on the LAN or WAN, initiate a connection over IPv4 or IPv6 to the DUT’s SMB or FTP server, and perform the steps documented in the test case procedure. If all steps in the procedure are successful the test will pass; if any step in the procedure is unsuccessful the test case will fail.

NOTE: CDRouter can be configured to use multicast DNS (mDNS) for address resolution by setting the testvar useMulticastDNS to yes. When this testvar is enabled, CDRouter will first attempt to use mDNS followed by DNS if the initial mDNS query is unsuccessful. By default, mDNS resolution is disabled.

DUT Address Resolution for LAN Side Storage Tests

CDRouter will first attempt to resolve a DNS A record for the DUT’s SMB or FTP server name. If this initial DNS query is not answered, CDRouter will attempt to resolve the server’s address via NetBIOS. If both the DNS and NetBIOS queries are unsuccessful, the test will generate a failure and abort.

CDRouter will attempt to resolve a DNS AAAA record for the DUT’s SMB or FTP server name. If this DNS query is not answered the test will generate a failure and abort.

In some cases the CPE may not support IPv6 address resolution via DNS, or in the case of a standalone NAS, it may not know the IPv6 address of the NAS. If this is true, the CDRouter configuration file should contain explicit IPv4 and IPv6 server addresses for each SMB and/or FTP user configuration that is defined. The IPv6 address of the DUT must be determined prior to testing.

Note that if a server name, as opposed to an address, is specified for any SMB or FTP user configuration, CDRouter will assume that that user configuration is supported over both IPv4 and IPv6. As a result, if the CPE does not support IPv6 address resolution via DNS, separate user configurations should be defined, one with an explicit IPv4 server address and another with an explicit IPv6 server address.

DUT Address Resolution for WAN Side Storage Tests

CDRouter will use the CPE’s WAN IP address, as specified by the testvar wanIspAssignIp, as the primary endpoint of the test. If the DUT is a standalone NAS on the LAN side of the CPE, a virtual service (also commonly referred to as a port forwarding rule) must be configured on the CPE. The virtual service ensures that the CPE properly forwards storage related protocol traffic received on the WAN to the DUT on the LAN.

Virtual services can be either manually configured by the tester, or automatically configured via another protocol such as UPnP. If the DUT requires UPnP for WAN access, the CPE’s UPnP functionality must also be enabled. Note that the use of UPnP may override any conflicting virtual services previously configured on the CPE. This can be problematic if a standalone NAS is connected to a CPE device that also supports access via the same storage protocol(s) from the WAN.

For example, if both the CPE and the standalone NAS support FTP access from the WAN, the standalone NAS may inadvertently override the CPE’s default WAN port 21 configuration via UPnP. This would effectively disable FTP access to the CPE’s integrated NAS from the WAN, which may not be the desired result.

CDRouter will attempt to resolve the DUT’s server address by looking up the server name in the list of user-configured DNS hostnames specified for the primary IPv6 DNS server, ipv6WanDnsServer, and the backup IPv6 DNS server, ipv6WanBackupDnsServer. If a DNS AAAA record is found for the configured server name, CDRouter will use the associated IPv6 address as the endpoint of the test.

If CDRouter is unable to internally resolve an IPv6 address for the configured server name, the CPE’s global WAN IPv6 address will be used as the target of the test. As a result, if WAN access to a standalone NAS is being tested over IPv6, one of the following must be true in order for CDRouter to target the correct endpoint:

1. An IPv6 address, rather than a name, must be configured for the DUT’s SMB or FTP server, OR

2. An IPv6 entry for the DUT’s SMB or FTP server name must be added to CDRouter’s internal DNS servers

CDRouter’s DNS servers can be configured with arbitrary IPv6 DNS entries by following the instructions in this Knowledge Base article.

Running

Storage support can be enabled within CDRouter by uncommenting the testvar supportsStorage and setting it to a value of “yes”. If testing storage related functionality over IPv6, IPv6 must also be enabled and and properly configured. Please see the CDRouter IPv6 User’s Guide for more information on configuring CDRouter for IPv6.

CDRouter Storage includes a number of test modules designed to verify a wide variety of NAS related functionality:

• smb.tcl
• ftp.tcl
• ftps.tcl
• ftp-wan.tcl
• ftps-wan.tcl
• smb-v6.tcl
• ftp-v6.tcl
• ftps-v6.tcl
• ftp-wan-v6.tcl
• ftps-wan-v6.tcl

Note that all FTP and FTPS related tests included with the CDRouter Storage add- on are performed in passive mode with support for extended passive mode (EPSV) enabled, unless otherwise noted in the test case description. Additional test cases for verifying basic functionality in passive mode with EPSV disabled, active mode with EPRT enabled, and active mode with EPRT disabled are included in each FTP/FTPS test module.

Licensing

To utilize CDRouter’s storage test capabilities, your license file must have the CDRouter Storage expansion enabled. In addition, if you plan to test storage functionality over IPv6 your license file must also have the CDRouter IPv6 add- on enabled.

For information on upgrading your license to support CDRouter Storage or CDRouter IPv6, please contact sales@qacafe.com. Please follow the instructions from support@qacafe.com when updating your license file to enable CDRouter Storage or CDRouter IPv6.

CDRouter will report the status of all available expansions during the installation process and during startup. To verify that CDRouter Storage and optionally CDRouter IPv6 are enabled on your system, execute the command cdrouter-cli -info as root and look for the lines “Storage is enabled” and “IPv6 is enabled”, as shown below. If these lines are present, CDRouter Storage and CDRouter IPv6, respectively, are enabled and ready to use.

Starting cdrouter-cli Fri Mar 25 13:28:42 EDT 2016
Copyright (c) 2001-2016 by QA Cafe
Version 10.0 build 1 (21410 trunk), built 2016-03-24 17:36:47 by nightly@cdr-forge6.lan (x86_64)
Loaded OS distro \S Kernel \r on an \m
Loaded OS version Linux-3.10.0-327.10.1.el7.x86_64 x86_64
Loaded Tcl version 8.6.4
Loaded buddy version 10.0.1
(builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) )
Trying to load modules from '.'
Trying to load modules from '/usr/cdrouter/tests'
Start command: /usr/cdrouter/bin/cdrouter-cli -info
Test Suite cdrouter 10.0.1
The system ID is 2df9e2a1f8c359183cf0191a20f2cc5a
Using license installed at: /etc/cdrouter.lic
Registered to: qacafe
Maintenance, Support and Upgrades until: 2016-05-11
Licensed to run: cdrouter
    Multiport   is enabled
    IPv6        is enabled
    Storage     is enabled
    IKE         is enabled
    TR69        is enabled
    TR69-EDM    is enabled
    Nmap        is enabled
    BBF.069     is enabled
    SNMP        is enabled
    Performance is disabled
CPU is Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz, bogomips 5599.98
Loaded TclXML version 3.1 (libxml2), TclDOM 3.0, xmldefs 3.1
Trying to load modules from '/usr/cdrouter/vendor/IOL/BBF.069/Tests'
BBF.069 version 1.2.075-11 (19093)

Configuration

Global Storage Configuration

CDRouter Storage has several configuration options, also known as test variables, or “testvars” for short, that define the DUT and the test environment.

The following tables describe all of the new testvars available with the CDRouter Storage expansion. The sections below group testvars by the protocol or functionality they define.

To globally enable storage support within CDRouter, the testvar supportsStorage must be set to a value of “yes”.

CDRouter Storage currently supports the verification of the SMBv1/CIFS, FTP, and FTPS protocols over both IPv4 and IPv6. FTP and FTPS are very similar and share the same CDRouter configuration. Information on the configuration of SMBv1/CIFS and FTP/FTPS is provided in the sections below.

SMB/CIFS User Configuration

The enable SMB support within CDRouter Storage, the testvar supportsStorageSmb must be set to a value of “yes”.

• supportsStorageSmb
• smbuser::server
• smbuser::share
• smbuser::folder
• smbuser::guest
• smbuser::user
• smbuser::password
• smbuser::permission

Of these seven testvars, the testvars folder, guest, user, and password are optional and may be omitted. The testvars server, share and permission are all required and must be included in each smbuser testvar_group definition.

Each smbuser testvar_group is independent and defines a unique test configuration. The server, share, and folder testvars specify the exact location of the resource under test, which is a folder on a network attached SMB server share. The guest testvar defines whether or not the resource under test allows guest access. The SMB server in this case is the DUT, and can be either integrated within the CPE or located in a standalone NAS on the LAN side of the CPE. The user, password, and permission testvars define how CDRouter should attempt to connect to and interact with the DUT while accessing the specified resource.

This type of configuration allows for a great deal of flexibility when testing storage functionality on a CPE and/or standalone NAS. Multiple user accounts with varying levels of access can be quickly tested with different SMB servers, shares, and folders within the same test run using the same CDRouter configuration file. SMB servers implementing user level and/or share level access controls can be tested with this style configuration.

NOTE: SMB/CIFS “guest” access can also be tested with the CDRouter Storage expansion. If guest access on the specified share is not allowed, the guest testvar should be set to “no”. In these scenarios CDRouter will connect to the SMB server using the username and password specified. By default CDRouter assumes that guest access is not permitted. If guest access is permitted for the specified share, the username and password testvars may be omitted. Testvars can be omitted by commenting them out, removing them entirely from the smbuser testvar_group definition, or leaving them empty (which is indicated by setting the testvar value to double quotes, “”). If the username and password testvars are omitted, CDRouter assumes the specified share allows guest access.

Other SMB/CIFS Configuration Options

Two additional SMB configuration testvars are also available:

• storageSmbMaxSessions
• storageSmbTimeout

FTP/FTPS User Configuration

The enable FTP and/or FTPS support within CDRouter, the testvars supportsStorageFtp and supportsStorageFtps must be set to a value of “yes”, respectively.

• supportsStorageFtp
• supportsStorageFtps

As with SMB/CIFS, up to 10 unique FTP/FTPS user configurations can be defined as ftpuser1 through ftpuser10 testvar_group instances. The format of each ftpuser testvar_group in the CDRouter configuration file is shown below:

# -- CDRouter config file format for a single ftpuser testvar_group 
testvar_group ftpuser1 {
  testvar server     myNAS
  testvar path       /home/qacafe
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadWrite
}

Each ftpuser testvar_group includes the following five testvars.

• ftpuser::server
• ftpuser::path
• ftpuser::user
• ftpuser::password
• ftpuser::permission

Of the five testvars listed above, only password is optional. All other testvars are required and must be included in each ftpuser testvar_group definition.

Each ftpuser testvar_group is independent and defines a unique test configuration. The server and path testvars specify the exact location of the resource under test which is a directory on a network accessible FTP server. The FTP server in this case is the DUT, and can be either integrated within the CPE or located in a standalone NAS on the LAN side of the CPE. The user, password, and permission testvars define how CDRouter should attempt to connect to and interact with the DUT while accessing the specified resource.

This type of configuration allows for a great deal of flexibility when testing storage functionality on a CPE and/or standalone NAS. Multiple user accounts with varying levels of access can be quickly tested with different FTP servers and directories within the same test run using the same CDRouter configuration file. FTP servers implementing user level and/or share level access controls can be tested with this style configuration.

NOTE: FTP’s concept of “anonymous” access can be tested by setting the testvars username and password to “anonymous”.

Other FTP/FTPS Configuration Options

Two additional FTP/FTPS configuration testvars are also available:

• storageFtpMaxSessions
• storageFtpTimeout

General Configuration Options

There are two additional configuration options which apply to the general setup of CDRouter Storage. These options define which file to use for all storage- related upload and download tests. By default CDRouter will use one of three included files, although a user specified file can also be configured.

• storageFileSize
• storageUserFilePath

Configuration Notes - Required User Permissions

Certain test cases require ReadOnly or ReadWrite permissions. CDRouter will automatically choose an smbuser or ftpuser with the required permissions at the start of each test. If no smbuser or ftpuser configurations exist with the required permissions for a specific test case, the test case will be skipped automatically by CDRouter. The table below lists the required permissions for each test case, by test module.

Configuration Notes - Scaling Tests

Three separate scaling tests are included in each CDRouter Storage test module. The first scaling test, with the suffix 50, is designed to verify that the DUT supports simultaneous access from the same smbuser or ftpuser configuration on the same LAN or WAN client. To perform this test CDRouter creates two instances of the chosen smbuser or ftpuser and attempts to establish a connection to the DUT. As a result, the source IP address of each SMB or FTP client created by CDRouter is the same.

Scaling test 51 is designed to verify each defined smbuser or ftpuser configuration. Each smbuser or ftpuser configuration is tested sequentially on the same LAN or WAN client. All SMB or FTP clients will have the same source IP address.

Scaling test 52 is designed to verify the maximum number of concurrent SMB or FTP sessions supported by the DUT. For this test CDRouter creates multiple instances of the chosen smbuser or ftpuser configuration on individual LAN or WAN clients. As a result, each SMB or FTP client that is created will have a unique source IP address. CDRouter creates up to storageSmbMaxSessions or storageFtpMaxSessions clients. If this number exceeds the size of the LAN DHCP pool, CDRouter will only create up to the maximum number of clients available within the pool.

Example Configurations

There is a wide variety of NAS configurations and functionality that can be tested with CDRouter Storage. A few basic example configurations for common storage testing scenarios are provided below.

Note that if a name is provided for the server testvar in any smbuser or ftpuser configuration, CDRouter assumes that access over both IPv4 and IPv6 is possible.

Example 1: Read-write SMB user

In this example a single SMB user configuration is defined. The DUT is an SMB server with the name MyNAS, which may be integrated into the CPE device or located in a standalone NAS device connected to the LAN side of the CPE. The DUT is configured to provide an SMB share named data to remote clients. A user named qacafe with the password qacafe123 is configured on the DUT with Read-Write access to the folder /home/qacafe on the data share.

In this configuration both FTP and FTPS access to the DUT is not supported or it is disabled. Additionally, it is assumed that the DUT supports up to five simultaneous SMB sessions, and all test files utilized by CDRouter will be 100 KBytes in length.

NOTE: Since no Read-Only users are defined in this configuration, all tests requiring Read-Only access will be automatically skipped by CDRouter.

NOTE: The smbuser configuration described in this example is applicable to DUT’s implementing user level access control for the user qacafe OR share level access control for the folder /home/qacafe on the share data. This same methodology is applicable to all of the examples presented in this section.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes
testvar_group smbuser1 {
  testvar server     MyNAS
  testvar share      data
  testvar folder     /home/qacafe 
  testvar guest      no
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadWrite
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 2: Read-only SMB user

This example is nearly identical to Example 1 above. The only difference is that the user qacafe has been configured with Read-Only access to the folder /home/qacafe on the share data. Since no Read-Write users are defined in this example, all tests requiring Read-Write access will be automatically skipped by CDRouter.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes

testvar_group smbuser1 {
  testvar server     MyNAS
  testvar share      data
  testvar folder     /home/qacafe
  testvar guest      no
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 3: Read-write SMB user with no folder

This example is nearly identical to Example 1 above. The only difference is that no folder has been defined. The user qacafe simply has Read-Write access to the share data.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes

testvar_group smbuser1 {
  testvar server     MyNAS
  testvar share      data 
  testvar guest      no
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 4: SMB user with read-write access to folder and read-only access to share

In this example the user qacafe is configured with Read-Write access to the folder /home/qacafe on the share data and Read-Only access to the share pictures. This is accomplished with two separate smbuser configurations. Since both Read-Write and Read-Only smbusers exist in the configuration file, no SMB-related tests will be skipped.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes

testvar_group smbuser1 {
  testvar server     MyNAS
  testvar share      data
  testvar folder     /home/qacafe 
  testvar guest      no 
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar_group smbuser2 {
  testvar server     MyNAS
  testvar share      pictures
  testvar guest      no 
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 5: Specify server by IP address

Typically an SMB server will be specified by name. However, in some cases the SMB server or CPE may not support resolution of the server address via DNS or NetBIOS. In these cases the server IP address should be explicitly configured instead. Only a single IPv4 or IPv6 address can be configured for each smbuser testvar_group definition.

In this example the server name has been replaced by IP address. In the smbuser1 configuration the user qacafe has Read-Write access to the folder /home/qacafe on the share data offered by the server 192.168.1.1. In the smbuser2 configuration the user matt has Read-Only access to the share pictures on the server 3001:dddd::1.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes

# --- IPv4
testvar_group smbuser1 {
  testvar server     192.168.1.1
  testvar share      data
  testvar folder     /home/qacafe
  testvar guest      no
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

# --- IPv6
testvar_group smbuser2 {   
  testvar server     3001:dddd::1
  testvar share      pictures
  testvar guest      no 
  testvar user       matt
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 6: Configure SMB access to two different servers

It is also possible to connect a standalone NAS to a CPE with integrated NAS capabilities and test both during the same test run using the same configuration file. This allows the individual NAS devices to be tested, and also ensures that they interact as expected when connected to the same network.

In this example the smbuser1 configuration defines a user mike with Read- Only access to the folder /home/mike on the share data offered by the server myCPE. In the smbuser2 configuration the user matt has Read-Write access to the folder media/videos on the share pictures provided by the server myStandaloneNAS connected to the LAN side of the CPE.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration

testvar supportsStorageSmb    yes

# --- User access to CPE
testvar_group smbuser1 {
  testvar server     myCPE
  testvar share      data
  testvar folder     /home/mike
  testvar guest      no
  testvar user       mike
  testvar password   qacafe123
  testvar permission ReadOnly
}

# --- User access to standalone NAS
testvar_group smbuser2 {
  testvar server     myStandaloneNAS
  testvar share      work
  testvar folder     media/videos 
  testvar guest      no
  testvar user       matt
  testvar password   qacafe123
  testvar permission ReadWrite
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Example 7: Read-write FTP user

In this example a single FTP user configuration is defined. The DUT is an FTP server with the name MyNAS that is configured to provide FTP access to the directory /home/qacafe. A user named qacafe with the password qacafe123 is configured on the DUT with Read-Write access to this directory.

In this configuration SMB access to the DUT is not supported or it is disabled. Additionally, it is assumed that the DUT supports up to 10 simultaneous FTP sessions, and all test files utilized by CDRouter will be 100 KBytes in length.

NOTE: Since no Read-Only users are defined in this configuration, all tests requiring Read-Only access will be automatically skipped by CDRouter.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    no

# -- FTP/FTPS configuration
testvar supportsStorageFtp    yes
testvar supportsStorageFtps   no

testvar_group ftpuser1 {
  testvar server     MyNAS
  testvar path       /home/qacafe
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadWrite
}

testvar storageFtpMaxSessions 10
testvar storageFtpTimeout     120000

# -- Other configuration options
testvar storageFileSize       100

Example 8: Read-only FTPS user

The configuration in this example is nearly identical to the configuration described in Example 7. The primary difference is that the DUT, myNAS, is configured for FTPS access only, and the user qacafe has Read-Only access to the directory /home/qacafe.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    no

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   yes

testvar_group ftpuser1 {
  testvar server     MyNAS
  testvar path       /home/qacafe
  testvar user       qacafe
  testvar password   qacafe123
  testvar permission ReadWrite
}

testvar storageFtpMaxSessions 10
testvar storageFtpTimeout     120000

# -- Other configuration options
testvar storageFileSize       100

Example 9: Read-only SMB share with guest access

This example is nearly identical to Example 2 above. The main difference here is that guest access is enabled on the share public. In addition, a second smbuser configuration is included to verify that the configured user qacafe can also access the guest share.

# Storage Configuration:
testvar supportsStorage       yes

# -- SMB configuration
testvar supportsStorageSmb    yes

testvar_group smbuser1 {
  testvar server     MyNAS
  testvar share      public
  testvar guest      yes 
  testvar permission ReadOnly
}

testvar_group smbuser2 {
  testvar server     MyNAS
  testvar share      public
  testvar guest      yes 
  testvar username   qacafe
  testvar password   qacafe123
  testvar permission ReadOnly
}

testvar storageSmbMaxSessions 5
testvar storageSmbTimeout     120000

# -- FTP/FTPS configuration
testvar supportsStorageFtp    no
testvar supportsStorageFtps   no

# -- Other configuration options
testvar storageFileSize       100

Testing Exercises

A great way to start testing a storage enabled CPE or standalone NAS device with CDRouter is to simply configure and run each of the applicable test modules included with the CDRouter Storage expansion individually. Analysis of these initial results will provide valuable insight and may help identify basic functional issues within the DUT that should be addressed.

The following sections describe some of the more common advanced testing exercises that are possible with CDRouter Storage.

Vary the IPv4/IPv6 Configuration

There are a number of different configuration scenarios that can be created with CDRouter which involve different combinations of IPv4 and IPv6 WAN and LAN modes and various configuration options. One simple yet powerful test exercise is to run the same set of storage tests while varying the underlying IPv4 and/or IPv6 configuration of the DUT. The DUT’s storage functionality should remain consistent across all of the IPv4 and IPv6 configuration scenarios supported.

Test with Wireless

The DUT’s storage related functionality should be consistent over both wired and wireless LAN interfaces. With CDRouter it is very easy to confirm this by adding at least one wireless LAN interface to your configuration file. In addition, storage functionality can be verified over all of the wireless modes and security schemes supported by the DUT.

Test with Different Disk Formats

NAS devices often support a variety of file system formats including FAT/FAT32, NTFS, HFS/HFS+, EXT2, EXT3, EXT4, etc. The file system format is a low-level attribute of a physical storage device that should be transparent to higher layer protocols that may use that device. A simple testing exercise is to verify that the DUT’s storage functionality and behavior over all of the file systems supported. With CDRouter Storage any inconsistencies due to the file system in use can be quickly identified.

Test with Different Timeout Values

CDRouter Storage includes configuration options for adjusting the SMB client timeout, storageSmbTimeout and FTP client timeout, storageFtpTimeout. Both timeouts are specified in milli-seconds and default to a relatively large value of 120000 (120 seconds).

These options are useful for ensuring that the DUT is responsive and capable of performing the desired operations within a practical amount of time. For example, if you expect the DUT to complete the desired operation(s) in no more than 10 seconds, you can set these timeout values to 10000. If the DUT takes longer than 10 seconds to complete any operation, CDRouter’s SMB or FTP clients will time out and the test will fail.

These options are also useful for adjusting the timeout when large user-defined files are configured using the storageUserFilePath testvar. Large user-defined files may take longer to upload and/or download than the default files used by CDRouter which may exceed the default SMB or FTP client timeout value of 120 seconds.

Verify Service Availability

Another good testing exercise is to verify that storage services are indeed unavailable when they are disabled within the CPE or NAS device’s management interface. Some implementations may continue to allow access to storage related services even if they are administratively disabled. With CDRouter this condition can be easily tested. Start by building a baseline configuration and test package for the various storage services supported by the DUT. Next, disable each storage service one at a time, and verify that all applicable tests generate failures. In this scenario test failures represent positive results, indicating that various aspects of the associated storage service are indeed disabled.

Test Different Users and Shares

With CDRouter Storage up to 10 unique SMB user configurations and 10 unique FTP user configurations can be defined. Including a variety of different user configurations in your CDRouter config file is a good way to verify the NAS functionality of the DUT. Many different scenarios involving different servers, users, folders or directories, and permissions can be easily tested with CDRouter Storage. Some simple examples are provided below:

• Create two different users with different permissions to the same share on the server. User1 may have ReadOnly permission, whereas User2 may have ReadWrite permission.

• Create two different shares with different access permissions. Share1 might have ReadWrite access for all users, whereas Share2 may have ReadOnly access for all users or for a specific group of users.

Duration Testing

The CDRouter UI allows for editing a package which includes a number of useful run-time options which can be utilized to ensure NAS functionality over long durations. For example, with CDRouter you can easily repeat the same storage test (or tests) 1000 times to ensure robustness and consistent behavior.

Verify “Guest” Access on SMB/CIFS Shares

Many SMB/CIFS implementations support the concept of “guest” access on specific shares. Shares with guest access typically do not require authentication, which allows any user to access the contents of the share. With CDRouter Storage guest access can be easily verified. In addition, CDRouter Storage can also be used to verify that configured users also have access to guest shares.

Possible Problems

Standalone NAS Devices and Unexpected Firewall Modifications via UPnP

Some standalone NAS devices have the ability to automatically create and modify port mappings on the CPE via UPnP. If the CPE supports UPnP and UPnP is enabled, NAS devices on the LAN may unexpectedly modify the firewall configuration of the CPE which can lead to confusing test results. For example, a NAS device connected to the LAN side of the CPE may open port 21 to allow FTP access from the WAN. Such a modification will impact firewall testing on the CPE and may conflict with the pre-existing configuration of the CPE.

To ensure that a NAS device is not altering the state of the CPE while testing, UPnP should be disabled. Of course in some scenarios it may be desirable to enable UPnP on the CPE and ensure that the NAS device properly modifies the firewall to allow remote access from the WAN. Both scenarios can easily be tested with CDRouter Storage.

If the CPE has directly attached USB attached storage with FTP access from the WAN enabled and a standalone NAS connected to its LAN, we recommend that UPnP be disabled when testing the directly attached storage device, and enabled when testing the NAS device attached to the LAN.

Storage Services May Not be Fully Initialized Following a Reboot

CPE devices that support directly attached USB storage may not properly initialize some or all storage related services following a reboot. This type of issue can be easily identified with CDRouter by creating a test package containing a baseline set of CDRouter Storage tests and running that test package back-to-back an arbitrary number of times. Adding a remote controlled power strip to the test setup will ensure that the CPE is automatically power cycled by CDRouter between each iteration of the test package.

Third Part Software May Limit Access

Some standalone NAS devices utilize third party, cloud-based applications or services to allow remote access to the NAS from the WAN. These applications make it very easy to share and access the NAS device with computers connected to the Internet. In situations where the primary WAN connection on the CPE is down, access to the NAS device may be inadvertently limited on the LAN as well as the WAN. A good testing exercise is to ensure that LAN side connectivity to the NAS device is still possible even if the WAN connection is down.