CDRouter Security User Guide

Introduction

The CDRouter Security expansion includes a number of features designed to test and evaluate the overall security of a CPE device. These features include port scanning tools, traffic analysis for known malware and threats, and parental controls features.

Licensing

CDRouter Security is a licensed expansion that must be purchased from QA Cafe. For information on upgrading your license to include CDRouter Security or any other expansions, please contact sales@qacafe.com.

CDRouter will report the status of all available expansions during the installation process and during startup. In addition, the list of installed expansions can also be displayed at any time by visiting the /system/info page within the web UI or by running the command cdrouter-cli -info as root from the command line.

The CDRouter Security expansion includes the CDRouter ICS and Nmap expansions which were previously licensed separately. Any existing licenses which include the ICS or Nmap expansions will retain this functionality. All new licenses must include the Security expansion to enable ICS or Nmap functionality.

System Requirements

The CDRouter Security expansion was first released with, and requires a minimum version of, CDRouter 12.0.

To utilize all of the features available with the CDRouter Security expansion an NTA1000v5 or newer platform is also required.

Components

• Internet Connection Sharing (ICS)
• Network Discovery and Port Scanning with Nmap
• Traffic Analysis with Suricata
• Parental Controls

Internet Connection Sharing (ICS)

Overview

CDRouter ICS enhances traditional closed loop testing by providing access to the internet or other outside networks for non-test traffic. This is accomplished by routing test traffic and network traffic separately, enabling testing for:

• Devices that require access to external resources or start-up procedures
• Devices that have cloud- or app-based management systems or user interfaces
• Devices that access real-time web applications as part of regular operations
• Devices that require access to CRLs or other certificate validation resources

Configuration

The following testvars control internet connection sharing within CDRouter:

• enableICS
• icsInterface
• icsShareIPv4
• icsShareIPv6

To enable internet connection sharing, the testvar enableICS must be set to “yes”. The testvar icsInterface must be set to the network interface on your CDRouter system which CDRouter ICS will use to route traffic to the internet.

The testvars icsShareIPv4 and icsShareIPv6 control whether internet connection sharing is enabled for IPv4 and IPv6 traffic, respectively. By default, both testvars are set to “yes” meaning internet connection sharing is enabled for both traffic types. To disable internet connection sharing for a traffic type, set that testvar’s value to “no”. Please note that enabling IPv6 internet connection sharing requires also enabling IPv4 internet connection sharing.

Test Methodology

CDRouter has traditionally been used for closed loop functional testing of CPE devices. In a closed loop setup, CDRouter’s LAN and WAN interfaces are connected directly to the CPE’s LAN and WAN interfaces, respectively. In this setup, the CPE alone is the device under test (DUT).

In certain situations an additional access concentrator may be required to terminate the CPE’s WAN interface. This occurs when the CPE’s WAN interface is not Ethernet and is instead LTE, DSL, DOCSIS, GPON, etc. In these situations a DSLAM, CMTS, or other access concentrator may be included in the closed loop setup.

In a closed loop setup, CDRouter controls all aspects of the test environment and provides end-to-end connectivity through the DUT for testing. CDRouter simulates the access network and all WAN servers with which the DUT communicates. This approach isolates the DUT and provides consistent and repeatable test results. Test failures in a closed loop setup can be traced directly to issues or functional problems with the DUT.

CDRouter ICS is an expansion that extends the traditional closed loop setup by providing Internet access to the DUT for non-test traffic. This makes it possible to test CPE devices that have cloud- or app-managed elements that require Internet access.

CDRouter ICS implements internet connection sharing by reconfiguring the iptables and ip6tables rules within the host’s operating system. internet connection sharing can be enabled independently for IPv4 and IPv6 traffic in most CDRouter configurations. CDRouter ICS also provides extended DNS functionality that allows requests for non-test resources to be answered by CDRouter.

IPv4 Internet Connection Sharing

When IPv4 internet connection sharing is enabled, CDRouter will create a simple NAT44 configuration on the system’s management interface at the start of the test run. When a packet is later received on the WAN, CDRouter will make a routing decision based on the destination IP address of the received packet.

Packets that have destination IP’s matching a known test stack will be processed by CDRouter as usual. All other packets will be forwarded by CDRouter to the management interface where they will be NAT’ed by the operating system and sent out on the corporate LAN.

IPv6 Internet Connection Sharing

IPv6 internet connection sharing works much the same way as IPv4 internet connection sharing - when enabled, CDRouter will create a simple NAT66 configuration on the system’s management interface. There are some additional caveats that apply to IPv6 internet connection sharing, namely that IPv6 internet connection sharing can only be enabled if IPv4 internet connection sharing is also enabled.

DNS

CDRouter ICS also includes enhanced DNS functionality to ensure that the DUT has seamless access to external resources.

In a typical closed loop setup, CDRouter’s DNS servers contain records for only a handful of static, well-known resources. Records are also added dynamically as needed during testing, and users have the option of defining additional records in the configuration file. CDRouter’s DNS servers are only able to provide answers to queries for known resources. As a result, queries for other external resources will go unanswered.

When internet connection sharing is enabled, CDRouter’s DNS servers will use the operating system’s DNS resolver when a query cannot be answered using its own records. The operating system may attempt to resolve queries locally via the /etc/hosts file before sending them to an upstream DNS server.

This additional functionality allows the DUT as well as its LAN clients to resolve external resources. Currently, this feature is only supported for queries for A, AAAA, CNAME, MX, PTR, SPF and TXT records.

Caveats

There are a number of caveats associated with internet connection sharing technique implemented by CDRouter ICS. Specifically:

• The CDRouter system’s management interface must have an IPv4 address and Internet connectivity in order for IPv4 Internet connection sharing to work. Likewise, the system must also have an IPv6 address and connectivity in order for IPv6 internet connection sharing to work.

• No ALGs are enabled within the NAT44 and NAT66 configuration applied by CDRouter to the management interface. Some protocols are not compatible with NAT or require an ALG if NAT is present. As a result, some non-test services or features required by the DUT may not be compatible with this technique.

• The IPv4 and IPv6 configuration of CDRouter’s primary WAN interface must not conflict with the IPv4 and IPv6 configuration of the management interface on the system. This requirement is imposed by the operating system when configuring NAT44 and NAT66 on the management interface. If encountered this requirement can be met by changing the IP addresses used by CDRouter on the primary WAN.

• Internet connection sharing has the potential to generate very large log and capture files if a significant amount of traffic is forwarded to the system’s management interface.

• Internet connection sharing is only enabled while CDRouter is running tests.

• Packets destined for addresses within CDRouter’s free network range, for both IPv4 and IPv6, will not be forwarded to the internet. As a result, some care must be taken to ensure that the free network range does not conflict with real servers or services on the internet that users may want to reach.

• Enabling ICS may impact test results if resources that are not normally accessible in a closed loop environment become accessible.

Testing Exercises

There are a number of interesting new test scenarios that are possible when internet connection sharing is enabled:

• The reporting capabilities of any cloud or app elements can be verified in real-time while tests are being performed. Information such as the overall status or health of the DUT, the number of connected LAN clients, availability of new firmware, etc. an be analyzed for accuracy.

• Diagnostic utilities built in to the DUT that rely on external resources can be tested. This includes well-known utilities such as ping and traceroute and also proprietary utilities that would not typically be available in a closed loop setup.

• Verify the behavior of the DUT while performing actions such as a firmware download while CDRouter renumbers the WAN interface.

• Test with and without internet connection sharing enabled to ensure that device operates properly if the internet and other external resources are not available.

Network Discovery and Port Scanning with Nmap

Overview

The CDRouter Security expansion includes a fully integrated and commercially licensed version of the popular open-source tool Nmap.

From the Nmap website:

Nmap (“Network Mapper”) is a free and open source utility for network discovery and security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. It was designed to rapidly scan large networks, but works fine against single hosts.

CDRouter utilizes Nmap to perform a series of standard scans targeting the DUT’s LAN and WAN interface. These scans are designed to probe the DUT with various network traffic in an attempt to discover open ports, fingerprint, and otherwise discover information about the DUT.

Configuration

CDRouter’s Nmap feature is very easy to use and is enabled by setting the testvar enableNmap to a value of yes. Three additional testvars control the behavior of Nmap:

• nmapPorts: the list or range of ports that are scanned by Nmap. This defaults to the full port range of 0-65535 but also accepts a keyword of default. When set to default, Nmap will automatically select the range for a scan based on commonly used ports. Large port ranges will increase both the scan time and the log files that are generated. Scanning the full port range is recommended.

• nmapScanTimeout: this testvar controls how long CDRouter will wait before stopping an Nmap scan. Under some conditions, this may need to be increased for long running scans. This value can also be reduced to ensure that scans complete quickly. If a timeout occurs during a scan, the test case will be marked as a FAIL.

• nmapTimingTemplate: this testvar is used to select the timing profile that will be used for each scan. Nmap has 6 built in timing templates that adjust retries and timeouts. The values range from 0 to 5. Lowering this value will produce slower scans.

Test Methodology

CDRouter includes four Nmap test modules that target the DUT’s LAN and WAN interface over IPv4 or IPv6. In addition, if the DOCSIS expansion is installed two additional Nmap test modules targeting the DOCSIS WAN interface are available:

These test modules all contain the same core set of 16 test cases, each executing a popular Nmap scan against the DUT. The 16 test cases included in each module and the corresponding Nmap scans are:

All Nmap tests produce standard CDRouter log files. The Nmap scan output for each test is captured and displayed in the corresponding test log. Nmap test results should be treated as informational only. CDRouter does not make pass or fail determinations based on the contents of Nmap scan output. Nmap tests will only produce a failure if the scan failed to run or if the scan timeout has been exceeded.

Users must manually inspect all Nmap test results to verify that scan output is acceptable. The output of a typical scan looks like this:

2020-02-25 19:10:00.459 SECTION(cdrouter-4812): Nmap output from scan:
    # Nmap 7.80 scan initiated Tue Feb 25 19:07:10 2020 as: nmap -n --min-rate 1000 -T5 -v --reason -oN - -Pn --max-parallelism 512 -sT -p 0-65535 192.168.1.1
    Warning: 192.168.1.1 giving up on port because retransmission cap hit (2).
    Nmap scan report for 192.168.1.1
    Host is up, received user-set (0.079s latency).
    Not shown: 49540 filtered ports, 15993 closed ports
    Reason: 49540 no-responses and 15993 conn-refused
    PORT     STATE SERVICE    REASON
    80/tcp   open  http       syn-ack
    443/tcp  open  https      syn-ack
    8080/tcp open  http-proxy syn-ack
    
    Read data files from: /usr/cdrouter/bin/../share/nmap
    # Nmap done at Tue Feb 25 19:10:00 2020 -- 1 IP address (1 host up) scanned in 170.29 seconds
2020-02-25 19:10:00.480 PASS: The nmap scan was successful
2020-02-25 19:10:00.484 PASS: Test v4_lan_tcp_connect_info (4812) passed

Nmap is a popular tool used widely for network probing and discovery. Running and analyzing Nmap scans during normal regression testing is an important part of assessing the overall security of a device. Nmap scan output provides essential information about a DUT, its behavior, and how it may be attacked or compromised in the field. Understanding how a device appears from an Nmap perspective is a critical part of ensuring that it is secure.

Traffic Analysis with Suricata

Overview

The CDRouter Security expansion includes a fully integrated version of the popular open-source network threat detection tool Suricata.

From the Suricata website:

Suricata is a free and open source, mature, fast and robust network threat detection engine.

The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing.

Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats.

With standard input and output formats like YAML and JSON integrations with tools like existing SIEMs, Splunk, Logstash/Elasticsearch, Kibana, and other database become effortless.

Suricata’s fast paced community driven development focuses on security, usability and efficiency.

Suricata provides capabilities including intrusion detection (IDS), intrusion prevention (IPS) and network security monitoring. It does extremely well with deep packet inspection and pattern matching which makes it incredibly useful for threat and attack detection. The project is backed by the Open Information Security Foundation (OISF) which provides long term protection for the openness of the code and helps to foster a community.

CDRouter utilizes Suricata to actively scan all non test traffic generated by the DUT in real-time while running standard CDRouter functional and performance tests. This new traffic analysis feature produces security alerts that are displayed in a new tab within CDRouter’s web UI.

Configuration

CDRouter’s Suricata based traffic analysis feature can be enabled by setting the enableSuricata to a value of yes. Note that this feature requires that internet connection sharing (ICS) also be enabled. Please see the ICS section for more information.

Test Methodology

When enabled, CDRouter will automatically launch Suricata and bind it to the ICS interface during start. Suricata is then used to analyze all non test traffic that is forwarded through the ICS interface. In most situations this will be traffic that is generated exclusively by the DUT for remote management and monitoring or other features that require cloud connectivity. However, it is important to note that all non-test traffic sent over the ICS interface will be analyzed by Suricata.

Security alerts may be generated by the DUT or by any non-CDRouter LAN clients connected to it. To pinpoint the source of a security alert all non-CDRouter LAN clients should be disconnected. This will indicate if the DUT or a LAN client is the source of the security alert. Individual LAN clients may then be connected until the source is identified.

The new Security Alerts tab should be viewed as an alternate test metric for the DUT during a specific test run. In addition to standard pass or fail test results, security alerts provide an indication of whether or not any unexpected, questionable, or otherwise generally bad traffic or behavior was observed by CDRouter during the test run.

Clicking on an individual alert within the Security Alerts tab will display more information about the rule that triggered it. Some contain references to a type of attack or to further documentation on a best practice. In addition, alert messages are displayed in test logs when and where they happened. CDRouter Security also generates packet captures of the analyzed traffic for deeper analysis.

The ideal outcome is therefore to have zero security alerts during every test run. Any security alerts that are generated should be analyzed and understood. It is important to note that zero security alerts does not guarantee that the DUT is secure; it only indicates that no unexpected, questionable, or otherwise generally bad traffic or behavior was observed by CDRouter during the test run.

Rulesets

Suricata is a signature based IDS; it scans traffic for known threats, malware, and security issues using signatures which are written using a well defined rule syntax. Collections of signatures are known as rulesets.

CDRouter ships with three rulesets by default and includes provisions for adding a fourth ruleset based on user defined custom rules. More information on these four rulesets can be found below.

The Suricata Ruleset

This is the default ruleset bundled with and used by Suricata. This ruleset contains approximately 300 rules that focus primarily on identifying malformed or invalid packets.

The Suricata project and ruleset is maintained by the Open Information Security Foundation (OISF). New Suricata rules will be automatically added to the Security expansion whenever the Suricata binary that is shipped with CDRouter is updated.

The Suricata ruleset is located in the following file on the CDRouter system:

/usr/cdrouter/etc/suricata/rules/engine.rules

Note that this file may be viewed but it must not be modified.

The ET Open Ruleset

The Emerging Threats ET Open ruleset is the industry standard open-source ruleset. This ruleset contains approximately 18,500 rules that focus on identifying known malware or malicious traffic.

This ruleset is maintained by proofpoint and is continuously evolving and updated daily. The ET Open ruleset is updated with every release of CDRouter and is located in the following file on the CDRouter system:

/usr/cdrouter/etc/suricata/rules/etopen.rules

Note that this file may be viewed but it must not be modified.

Proofpoint also offers an additional ruleset known as ET Pro which requires a license. Users with a valid ET Pro license should contact support@qacafe.com for information on how to install and use the ET Pro rules with their CDRouter system.

The CDRouter Suricata Ruleset

The CDRouter ruleset was developed by QA Cafe and contains a mix of rules based on CPE security best practices and knowledge of the test environment that is obtained from the CDRouter config file.

The CDRouter ruleset is dynamically generated at run-time and is tailored to the specific device that is being tested and the test environment. The CDRouter ruleset is updated periodically by QA Cafe as new security best practices and/or interesting rules become apparent.

CDRouter’s ruleset currently includes rules for detecting:

• FTP traffic on the WAN
• TLS sessions on the WAN that do not negotiate to TLSv1.2 or higher
• SSL/TLS records on the WAN that are not TLSv1.2 or higher
• Outbound unencrypted HTTP traffic on port 80
• New, inbound unencrypted HTTP connections on port 80
• Outbound traffic on port 443 that is not TLS
• Outbound unencrypted traffic on port 443
• Expired SSL/TLS certificates on the WAN
• IPv4 traffic that is not NAT’d
• The MAC address(es) of CDRouter’s LAN clients in plaintext on the WAN
• The DUT’s wifi SSID(s) in plaintext on the WAN
• The DUT’s wifi password (WPA/WPA2/WPA3) in plaintext on the WAN
• The DUT’s WAN MAC address in plaintext on the WAN

The CDRouter ruleset cannot be viewed since it is generated dynamically.

The User-Defined Custom Ruleset

Users also have the option of creating and using custom Suricata rules within CDRouter. This may be helpful for modeling specific deployment scenarios or identifying other anomalous traffic patterns or behavior.

All Suricata custom rules must adhere to the following guidelines and conventions:

• All custom rules must be contained in the /usr/cdrouter-data/rules/custom.rules file.

• All custom rules must follow Suricata’s documented Rules Format. and contain a valid action, rule header, and rule options.

• The only action supported by CDRouter is alert. As a result, all custom rules must use the alert action.

• The following Meta Keywords must be included in the options portion of all custom rule: msg, classtype, sid, rev.

• Additional rule option meta or non-meta keywords may also be used. Option keywords must be ordered according to the following convention:

  <msg> <classtype> <additional options> <sid> <rev>
  

• Custom rules must include unique signature IDs (defined by the sid meta keyword) in the range 1000000-1999999 as recommended in the Emerging Threats SID allocation guidelines.

• The classtype meta keyword must be one of the default Suricata classifications defined in /usr/cdrouter/etc/suricata/classification.config. Note that all classtype values have an implicit priority, which can be overridden using the priority meta keyword.

• The ruleset that is displayed for each alert within CDRouter is automatically extracted from the msg meta keyword in the rule definition. The first two words of the msg keyword must be capitalized and separated by a space. These two words will be displayed as the ruleset. Note that the first word must not contain any numbers; the second word may contain numbers. An empty string will be displayed as the ruleset for any rule definition that does not follow this format.

Suricata Configuration

Suricata has its own configuration file which CDRouter dynamically configures at run-time. Most Suricata configuration options are unmodified by CDRouter and left to default values. One exception is the Suricata HOME_NET configuration variable which is explicitly configured by CDRouter based on a list of well-known WAN networks including wanIspIp/wanIspMask, ipv6WanIspIp/ipv6WanIspPrefixLen and dhcpv6WanAssignPrefix/dhcpv6WanAssignPrefixLen.

DNS Traffic

If the DUT is configured to use CDRouter’s WAN DNS servers, CDRouter will respond directly to all DNS requests from the DUT, if possible. If CDRouter is unable to respond directly, the request will be proxied to the system resolver.

The system resolver will perform the lookup and send the answer back to CDRouter which will proxy it back to the DUT. Suricata has no visibility into DNS traffic generated by the DUT in this configuration since the DNS traffic is never sent over the ICS interface.

If instead the DUT is configured to use alternate non-CDRouter DNS servers, all requests will flow through the ICS interface just as any TCP or UDP traffic would. In this configuration Suricata will have visibility into DNS traffic generated by the DUT and may generate security alerts based on DNS signatures.

Requirements

CDRouter’s Suricata based traffic analysis feature requires internet connection sharing (ICS). ICS must be enabled and properly configured such that the DUT has IPv4 and/or IPv6 internet connectivity before Suricata can be enabled using the enableSuricata testvar.

ICS functionality requires an NTA1000 system with internet connectivity via the designated management interface or another unused interface. Note that ICS functionality is not supported in all CDRouter configurations.

Please see the ICS section of this user guide for more information.

Caveats

The following caveats apply to CDRouter’s Suricata based traffic analysis feature:

• Traffic analysis is only available on NTA1000 systems running CDRouter 12.0+.

• Traffic analysis requires that the DUT have IPv4 and/or IPv6 internet connectivity using CDRouter’s ICS feature.

• CDRouter’s ICS feature is only supported for native IPv4 and IPv6 WAN modes. Configurations utilizing tunneling or transition mechanisms do not support ICS or traffic analysis as a result.

• All traffic analysis security alerts and associated reporting features are available via CDRouter’s web UI only.

Testing Exercises

Enable Suricata for all test runs

Functional testing may trigger the DUT to contact cloud services. For example, a DHCP renumbering test on the WAN may result in the DUT contacting a cloud service to update its public IP information. As a result, Suricata should be enabled for all test runs using a variety of configurations and tests to maximize the likelihood of observing interesting behavior exhibited by the DUT.

In addition, enabling Suricata on longer duration test runs is also recommended. Enabling Suricata during a test run that lasts only 10 minutes may not be a large enough time window to observe anything meaningful.

Model deployment to ensure new and old services are working as expected

Suricata rules are built using a standardized, easy to use and documented signature language. Since CDRouter Security is passively monitoring all of the live network traffic generated by the DUT, rules can be created based on features and functionality specific to a certain deployment to make sure the expected services and resources are working correctly.

In addition to writing your deployment specific security checks, rules that flag certain policy behavior can also be written and added. For example, a device with a hard-coded URL to a cloud service that changed should be checked to ensure that the new firmware does not communicate with the old server.

Parental Controls

Overview

The term ‘parental controls’ is not standardized or well defined and generally refers to a set of features that collectively allow people to restrict internet access to specific devices on the LAN.

These features are often used by parents to limit their children’s access to the internet and control the content they are allowed to consume. Parental controls are an important security component in modern devices and a key differentiator for device manufacturers, service providers, and end users.

Common Parental Control Features

Most parental controls features fall into one of two main categories:

• Internet access restrictions
• Content filtering

Internet access restrictions are often time based and are designed to limit when users have access to the internet. Content filtering is much larger in scope and is designed to limit what internet resources users have access to.

Internet Access Restrictions

Internet access restriction features act like an on/off switch, allowing some or all clients and users to access the internet only during certain periods of time or under certain circumstances. The most common types of internet access restrictions are:

• Internet schedules
• Internet time limits

Internet schedules define blocks of time in which specific clients are allowed or denied access to the internet. Internet schedules may be unique per client and/or day of the week, making it possible to restrict internet access in a variety of ways. For example, a specific client (like Tina’s phone) could be allowed access to the internet only between 2:00 pm and 10:00 pm on weekdays; or all clients could be denied access to the internet every Saturday and Sunday.

Internet time limits define the maximum amount of time per day or week that a client is allowed access to the internet. Once this time limit has been reached, access is denied until the next cycle begins. For example, certain clients (like Fred’s laptop and Bob’s gaming console) could be allowed a maximum of two hours of internet access per day.

Content Filtering

Content filtering is a complex space that is typically used to prevent users and clients from accessing inappropriate websites or services and/or block malicious traffic.

Content filtering can take many forms, including:

• Blocking specific websites using URL filtering
• Blocking certain regions based on geo-IP information
• Blocking certain IP ranges based on IP reputation information
• Blocking specific applications, ports and protocols
• DNS-based domain classification filtering
• Blocking based on traffic classification obtained via deep packet inspection (DPI)

Many devices allow a combination of both time based internet access restrictions and content filtering rules to be applied. In addition, rules can be applied to all LAN clients or only to specific LAN clients which are uniquely identified by MAC address or other fingerprinting techniques.

Test Coverage

The CDRouter Security expansion currently contains test modules for verifying internet schedule functionality. Test modules for other common parental control features will be added in future releases.

Internet Schedule Testing

Two new test modules, internet-schedule.tcl and internet-schedule-v6.tcl, are available for verifying simple internet schedules. These modules verify that new sessions are either allowed or denied based on the configured schedule using HTTP/HTTPS traffic over IPv4 and IPv6, respectively.

A simple internet schedule is one in which each day of the week is divided into a maximum of three time blocks where internet access alternates between allowed and denied for the applicable client(s). For example, a simple internet schedule for one specific client on a specific day may look like this:

This same client may have the same or different schedules for other days of the week:

Each test module contains seven tests, one for each day of the week, named [ipv6_]internet_schedule_<day-of-week>. For example, the IPv4 test for Monday is named internet_schedule_monday and the IPv6 test for Wednesday is named ipv6_internet_schedule_wednesday. These tests verify that the internet schedule configured for the current LAN stack (ie LAN client) matches that stored in a number of new testvars, as described in the following sections.

Configuration

• internetScheduleBase: this testvar determines the base time from which the internet-schedule.tcl and internet-schedule-v6.tcl test modules calculate the first time which falls on a certain day of the week and which matches the internet schedule for the current LAN client in the time zone determined by the testvar internetScheduleTimeZone. For example, assume that the testvar internetScheduleTimeZone is set to Etc/UTC and the current LAN client has the testvar internetScheduleMondayMode set to allow and the testvar internetScheduleMondayTimes set to 9:00am - 5:00pm. If internetScheduleBase is set to 2007-06-02 11:00:00 UTC (Sat), the internet_schedule_monday testcase would calculate 2007-06-04 09:00:00 UTC (Mon) as the first Monday at 9:00am after the base time during which the LAN client should be allowed internet access. If set to the default value auto, the current time on the CDRouter system is used as the base time.

• internetScheduleTimeZone: the time zone configured on the DUT, specified as a valid TZ database time zone in the format Region/Zone (for example America/New_York). This testvar is configured in the main testvar group. If set to the default value auto, the timezone configured on the DUT is assumed to be the same as time zone configured on the CDRouter system.

• internetScheduleBlockedText: the text which should be found in the HTTP response received by the LAN client if its internet access was blocked. This testvar is only required if the DUT’s parental controls mechanism utilizes a proxy which returns valid web pages containing administratively blocked text for blocked LAN clients. This testvar is configured in the main testvar group.

• internetSchedule<day-of-week>Mode: there is one of these for each day of the week. Its value can be allow, deny or none. If set to none, the DUT has no schedule for the given LAN client for that day of the week. If set to allow or deny, then the tests are run and the time ranges specified dictate when the LAN client should be allowed (allow) or denied (deny) internet access. These testvars can be configured separately for each LAN testvar group (main, lan2, lan3, etc).

  • internetScheduleSundayMode
  • internetScheduleMondayMode
  • internetScheduleTuesdayMode
  • internetScheduleWednesdayMode
  • internetScheduleThursdayMode
  • internetScheduleFridayMode
  • internetScheduleSaturdayMode

• internetSchedule<day-of-week>Times: Again, there is one of these for each day of the week. Its value should be a time range such as 9:00am - 5:00pm which dictates when the LAN client should be allowed or denied internet access. These testvars can be configured separately for each LAN testvar group (main, lan2, lan3, etc).

  • internetScheduleSundayTimes
  • internetScheduleMondayTimes
  • internetScheduleTuesdayTimes
  • internetScheduleWednesdayTimes
  • internetScheduleThursdayTimes
  • internetScheduleFridayTimes
  • internetScheduleSaturdayTimes

The table below shows how CDRouter can be configured to test a few example Saturday internet schedules using the testvars internetScheduleSaturdayMode and internetScheduleSaturdayTimes:

Test Methodology

The internet schedule tests work by calculating a set of test times at which, based on the schedule of the current LAN client, internet access for the LAN client should be allowed or denied. Three test times are calculated for each time block: (time block start + 1 minute), middle, and (time block end - 3 minutes). This allows the behavior at the beginning, middle, and end of each time to be verified.

For example, if the the internet schedule for a specific client indicates that access should be allowed between 8:00 am and 6:00 pm on Saturday, the test times for Saturday will look like this:

For each of these test times, the test reconfigures CDRouter’s WAN NTP servers with the test time, reboots the DUT using the command configured via the RestartDut testvar and waits for an NTP request to be received by one of CDRouter’s WAN NTP servers. Once the DUT has requested NTP, it should be configured with the test time returned in CDRouter’s NTP response.

As an additional sanity check the test performs an HTTP GET to http://lanIp:dutMgmtPort, parses the Date header in the HTTP response and checks if the time reported by the DUT is within 15 seconds of the expected test time. The test then performs an HTTP and HTTPS GET from the LAN client to remotehost.cdroutertest.com, either via IPv4 or IPv6, and ensures that the request passes or fails as appropriate. This basic same procedure is repeated for each test time required to validate the internet schedule for that particular day of the week.

Requirements

In order to properly run the internet-schedule.tcl and internet-schedule-v6.tcl test modules, the following requirements must be met:

• The DUT must be configured to use CDRouter for time synchronization. This can be accomplished by either manually configuring the DUT to use CDRouter’s NTP servers, or by configuring the DUT to learn its NTP servers automatically via DHCP/DHCPv6 options on the WAN. Note that CDRouter will automatically provide NTP server information via DHCP/DHCPv6 options if requested by the DUT.

  For manual configuration of NTP, the hostname and IP address of CDRouter’s IPv4 NTP servers are defined by the following testvars:

    testvar ntpServer1                       3.3.3.6
    testvar ntpServerName1                   time.nist.gov
    testvar ntpServer2                       3.3.3.7
    testvar ntpServerName2                   time.foo.com
  

  Likewise, CDRouter’s IPv6 NTP servers are defined by the following testvars:

    testvar ipv6NtpServer1                   4321::23
    testvar ipv6NtpServerName1               time.nist.gov
    testvar ipv6NtpServer2                   4321::24
    testvar ipv6NtpServerName2               time.foo.com
  

• The DUT’s configured time zone must match the value specified by the testvar internetScheduleTimeZone. If there is a time zone discrepancy the internet schedule tests will not run properly.

• The RestartDut and RestartDutDelay testvars must be enabled and configured to run a script that will reboot the DUT. A reboot is one way of forcing the DUT to synchronize its time via NTP. Having the ability to consistently change the DUT’s time via NTP is key to how these tests operate. Without time changes via NTP these tests would take one full week to run.

• The lanIp and dutMgmtPort testvars should be properly configured. These tests perform a simple sanity following every NTP time change to verify that the DUT has learned and applied the new time. This check involves sending an HTTP GET to the DUT’s management URL at http://lanIp:dutMgmtPort, parsing the Date header, and verifying that it is within 15 seconds of the expected time. A warning will be displayed if this check fails or is unsuccessful, indicating that there may be an NTP synchronization issue.

Caveats

The internet-schedule.tcl and internet-schedule-v6.tcl test modules are subject to certain caveats:

• Only the behavior of new sessions are verified. Test cases for verifying the behavior of established sessions that cross schedule boundaries will be included in a future release.

• Failure to resolve a domain name via DNS is considered acceptable when determining if internet access has been denied. If DNS is successful, only the behavior of web traffic, specifically HTTP and HTTPS, is verified. No other types of traffic are used or verified as being allowed or denied based on the configured internet schedule.

• Internet schedules may be defined and applied globally to all clients but are more often defined and applied individually to specific clients. To define an internet schedule for a specific client, that client’s MAC address must be known, and must not change. To test an internet schedule for a specific client, the testvar lanMacId must be enabled and configured with that client’s MAC address.

• CDRouter allows internet schedules to be defined uniquely per LAN interface. If three LAN clients with three different internet schedules are defined, each test must be repeated three times to verify each internet schedule within one test run.

• It is important to ensure that configured internet schedules do not interfere with CDRouter’s ability to start and run test packages. For example, CDRouter’s LAN client(s) may not function properly if they are subject to an internet schedule imposed by the DUT. This can lead to confusing, inconsistent, and unexpected behavior and test results. For this reason it is advisable to run the internet schedule tests in an isolated package using specific clients.

• These tests take many minutes to run. This is because many reboot cycles are required to change the DUT’s time within each test. These tests would take one full week to complete without the reboot cycles, so a significant time savings has been achieved.

CPE Security Testing and Best Practices

Security touches all parts of the test process. This includes tests to verify firewalls and vulnerabilities, stress testing, performance testing, validation of configurations, and code inspection. Almost all development and test process can be evaluated from a security perspective. It is import to think of security as a process goal and understand how all test activities impact security.

The information below will help you think about the various aspects of your development and test process that impact security and how to leverage CDRouter to improve the security of your devices.

It starts with change

To make a robust and secure device, you’ll need to accept that product updates need to occur on a frequent basis. This includes both application and OS level updates. Today’s devices are complex and built upon a large ecosystem of open source technologies. These underlying technologies are constantly evolving (and need patching!). When a zero day security update occurs, you’ll need to update your product as quickly as possible.

Test Tips:

• Make sure you are testing your upgrade process early and often.
• Don’t forget to verify the downgrade path as well.
• You’ll need test automation in order to make release verification manageable.
• CDRouter offers upgrade and downgrade testing using TR-069.
• CDRouter can verify device behavior after upgrades and downgrades.

Know your ecosystem

Today’s products contain many key components and it is important to know which components and libraries you are using. There is really no excuse for not knowing what TLS library you are using and its current revision. Often, security concerns and issues are first reported at the individual project level before distributions are updated.

Test Tips:

• While source code level identification is the preferred method, CDRouter’s Nmap scanning tests can provide some feedback on open services and versions. Try running both v4_lan_os_detection_version and v6_lan_os_detection_version test cases.

For example, a LAN side Nmap scan may produce the following list of open services. A quick check on the Dropbear project pages reveals that this component is very out of date and that several CVE alerts contain references to Dropbear.

   PORT      STATE    SERVICE     REASON         VERSION
    21/tcp    open     pwdgen      syn-ack ttl 64 pwdgen
    22/tcp    open     ssh         syn-ack ttl 64 Dropbear sshd 0.46 (protocol 2.0)
    23/tcp    open     telnet      syn-ack ttl 64
    53/tcp    filtered domain      no-response
    80/tcp    open     http        syn-ack ttl 64 micro_httpd
    139/tcp   open     netbios-ssn syn-ack ttl 64 Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
    443/tcp   open     ssl/http    syn-ack ttl 64 micro_httpd
    445/tcp   open     netbios-ssn syn-ack ttl 64 Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
    30005/tcp open     unknown     syn-ack ttl 64
    44401/tcp open     unknown     syn-ack ttl 64

Know your default device behavior

Many vendors enter the device market building on existing distributions. They have little idea of the default behaviors built into their devices. Vendors need a deep understanding of their devices to know if specific security vulnerabilities impact them and also to know if any fixes change their device behavior.

Test Tips:

• Use CDRouter to understand your baseline functional behavior.
• Include functional and performance tests in this assessment.

Testing for memory fragmentation and memory leaks

System level issues involving memory can lead to complete system failure. Most devices contain OS and application level diagnostics that help expose memory fragmentation and leak issues. These are typically the first choice for testing automation.

Besides direct diagnostics, increasing the number of test iterations can help expose underlying issues with memory. CDRouter test cases provide the building blocks for long duration test runs. By adding looping into your CDRouter test package, you can create test runs with many iterations to help expose memory or other functional issues.

Test tips:

• Add looping test runs to extend test runs from hours to days (one CDRouter customer had a test run of continuous TR-069 tests running for a month).
• Select test cases across all the major functional areas of your device, including scaling, performance, TR-069, DHCP, DNS, USP, etc.

Running specific security tests

CDRouter has many test cases directly related to security. These may be based on an existing CVE or checking for TLS cipher suites. To make it easier to run these tests, CDRouter has a built in Security Test List. You can simply build a test package using the Security test list and then try running these tests.

Test tips:

• Create a new test package using the security test list.
• Some test cases may require additional CDRouter configuration settings in order to run.

Keep your ports closed

Open TCP or UDP ports on the WAN are an instant way to fail any security verification checks. However, the security focus has moved to the LAN side as well. Unsecured services on the LAN are now considered attack points.

CDRouter’s NMap expansion is a quick way to explore what services are visible on both the WAN and LAN. The OS detection tests reveal what services can be discovered when scanning on both the WAN and LAN.

Test tips:

• Use CDRouter NMap to find services on the LAN and WAN
• Run both IPv4 and IPv6 scans
• Run scans using multiple WAN protocol configurations.

Passwords

The topic of password selection is deserving of its own write up. For years, CPE devices have used canned passwords and made the end users responsible for changing them. Since most end users never do change them, the default password has become its own attack vector. Test engineers often disable security measures or use other weak passwords in order to “get stuff done”. The awareness around default passwords is starting to change. California recently passed new legislation to require unique secure passwords. That is a step in the right direction.

https://motherboard.vice.com/en_us/article/mbd5m4/california-is-making-it-illegal-for-devices-to-have-shitty-default-passwords

WiFi Defaults

Many vendors choose WiFi default settings to be the most compatible at the risk of security. Both WPA and TKIP encryption are now legacy protocols with known problems and most connected devices support both WPA2 and AES.

WiFi passwords should also default to unique values.

Test tips:

• Use CDouter’s wifi.tcl module to very TKIP is disabled
• Use CDRouter’s wifi.tcl module to verify WPA is disabled

Turn off UPnP by default

Many UPnP implementation have been plagued with vulnerabilities. A search of UPnP in the CVS database currently turns up over 100 different CVEs. Most modern applications are able to fully function without using UPnP. Vendors have started to leave UPnP disabled by default to avoid these vulnerabilities.

Test tips:

• Verify that UPnP is disabled running the upnp.tcl module
• Also check UPnP for IPv6

SSL certificate testing

Certificate testing is very challenging for even the most experienced testers. Often, SSL failures do not report specific error messages back to the user making it hard to know what went wrong. It is import to verify both the positive and negative test paths. What is the user experience when an expired certificated is encountered?

Test tips:

• Use the NTP offset testvar to set your devices clock in the future or past
• Verify certificate failure occurs as expected
• Try importing your own certificates for TR-069 or USP (can we do this for USP?)
• Test TR-069 with specific TLS versions
• Make sure common name validation is enabled and working
• TLS 1.2 or higher should always be used
• Make sure strong ciphers are supported

Don’t Forget IPv6

When it comes to security, IPv6 is often overlooked. Many existing exploits can be launched using either IPv4 or IPv6 and it is important to test with both protocols. When edge devices first started to enable IPv6, the firewall behaviors often lagged behind IPv4.

Test tips:

• CDRouter offers firewall tests for both IPv4 and IPv6
• CDRouter contains application tests for both IPv4 and IPv6.