Apple’s HomeKit is a framework for communicating with and controlling connected accessories within the home. HomeKit makes it possible for app developers, device manufacturers, and enthusiasts to create highly integrated IoT products and home automation solutions utilizing the HomeKit Accessory Protocol (HAP).
The HomeKit-enabled router plays a critical role in the HomeKit ecosystem by providing connectivity to all of the accessories in the home and the iOS or macOS apps used to control them. To address the specific requirements of the HomeKit-enabled router in a HomeKit environment Apple has developed the Apple HomeKit-enabled Router Certification program.
The HomeKit add-on for CDRouter provides fully automated testing of the Apple HomeKit-enabled Router Certification Test Plan. CDRouter utilizes Apple’s HomeKit Accessory Tester (HAT) along with various HomeKit enabled LAN clients to verify that the HomeKit-enabled router supports all of the discovery and security features defined within the test plan.
HomeKit-enabled routers that pass these tests may be subsequently certified and listed by Apple as such. For more information on the certification process, please contact Apple or firstname.lastname@example.org.
The HomeKit add-on for CDRouter must be purchased from QA Cafe. For information on upgrading your license to include the HomeKit add-on or any other CDRouter add-ons, please contact email@example.com.
CDRouter will report the status of all available add-ons during the installation
process and during startup. In addition, the list of installed add-ons can also
be displayed at any time by visiting the
/system/info page within the web UI
or by running the command
cdrouter-cli -info as root from the command line.
To perform the Apple HomeKit-enabled Router Certification Test Plan with CDRouter, users must have the following:
An NTA1000 based CDRouter system running CDRouter 12.3 or newer. Users should upgrade to the latest available release of CDRouter before performing any HomeKit tests.
A valid CDRouter license with the HomeKit and Multiport add-ons enabled. The Security add-on is optional and recommended when testing HomeKit-enabled routers that have cloud management features.
A HomeKit-enabled router; this is the device under test (DUT).
A satellite node if the HomeKit-enabled router supports mesh features. If the DUT does not support mesh features any test that verifies mesh functionality will be automatically skipped.
An Apple computer running macOS 10.15 (Catalina) or newer.
A copy of the HomeKit Accessory Tester (HAT), version 5.5.2 (3D714) or newer. HAT must be installed on the Apple computer and is a required part of the overall test setup. HAT can be obtained from Apple directly. For assistance please contact firstname.lastname@example.org.
A copy of the HAT firewall rules. These are distributed as a bundle of JSON based rule files and can be obtained from Apple directly. For assistance please contact email@example.com.
A copy of the Apple HomeKit-enabled Router Certification Test Plan. This can be obtained from Apple directly. For assistance please contact firstname.lastname@example.org.
A mechanism for configuring the DUT. In most cases this will be a vendor supplied iOS app. However, any management interface that supports general configuration of the DUT and that provides access to the HomeKit setup code will suffice.
The Apple HomeKit-enabled Router Certification Test Plan includes over 50 IPv4 specific test cases that are designed to verify that the DUT meets all of the functional requirements for a HomeKit-enabled router as defined by Apple. The test plan includes a mix of automated, semi-automated, and manual tests that focus on:
Pairing with the HomeKit-enabled router
Management and configuration of the HomeKit-enabled router
Discovery via mDNS and Bonjour of HAP enabled WiFi and Ethernet LAN clients that are connected to the HomeKit-enabled router
The application and verification of custom firewall rules on the Homekit-enabled router to verify various protection modes for HAP enabled LAN clients
Three protection modes are defined:
Auto: This mode allows a HomeKit controller to dynamically restrict the access of HAP enabled LAN clients to the WAN or other LAN clients by traffic type (TCP, UDP, or ICMP) and port.
Restricted: This mode allows a HomeKit controller to restrict specific HAP enabled LAN clients from accessing the WAN or other clients on the LAN.
No Restriction: This mode allows a HomeKit controller to provide a specific HAP enabled LAN client full, unrestricted WAN and LAN access.
HomeKit Accessory Tester (HAT)
To perform these tests CDRouter communicates with an instance of the HomeKit Accessory Tester, or HAT. HAT is a tool published by Apple that emulates a HomeKit controller such as an iOS app. HAT provides an API that can be scripted and used to discover, query, and control HAP enabled devices on a network.
By sending API commands to HAT, which is paired with the DUT, CDRouter is able to modify the DUT’s configuration by emulating a user making changes via an iOS app. Once these changes have been applied by HAT, CDRouter is able to verify that the DUT performed the requested actions by sending various types of traffic to and from HAP enabled clients on the LAN.
CDRouter must send properly formatted JSON encoded firewall rules to HAT to configure the DUT’s firewall. These rules contain all of the information required to configure the firewall for a specific test case or scenario as defined in the test plan. All individual rules have been pre-defined and are available as a collection of static rule files from Apple.
HAT must be installed on an Apple computer running macOS 10.15 (Catalina) or newer and must run as root. This computer must be connected to the LAN side of the DUT via Ethernet or WiFi.
HAT must also be paired with the DUT prior to testing. Once paired, HAT should
be used to send a timed write for the
Managed Network Enabled field with the
value “1” to the DUT. This enables the DUT to be configured via HAP.
On the computer running HAT, the JSON rules files needed for the tests should be placed in a directory uncompressed. The absolute path to this directory must be configured in CDRouter using the homeKitHATRulesPath testvar.
Basic Test Setup
The basic test setup for the HomeKit add-on is shown below. This setup is used to verify the behavior of the HomeKit-enabled router (the DUT) when two HAP-enabled WiFi LAN clients are directly connected to the primary AP. This setup verifies WiFi to WiFi client behavior specifically.
CDRouter’s WAN is connected to the DUT’s WAN via Ethernet. CDRouter is configured to terminate the DUT’s WAN using any of the available WAN modes supported by the DUT. If the DUT requires cloud connectivity, CDRouter’s ICS feature may also be enabled.
CDRouter’s primary LAN interface and the Apple computer running HAT are both configured to connect to the DUT via WiFi. There are no constraints on the WiFi configuration of the DUT; the user is free to choose any mode and security configuration. Note that the user must manually connect the Apple computer to the DUT before running any tests with CDRouter.
While running HomeKit tests CDRouter will communicate with HAT via its API. HAT will then configure the DUT, and CDRouter will verify all configuration changes using simulated HAP-enabled clients on the LAN.
Wired Test Setup
The wired test setup for the HomeKit add-on is shown below. This setup is similar to the basic test setup with the addition of a satellite/mesh node which has a wireless backhaul connection to the primary AP. This setup is used to verify wired Ethernet to Ethernet client behavior specifically.
A second Ethernet LAN interface is enabled and configured within CDRouter. This interface is connected to the wired LAN port on the satellite/mesh node.
Mesh Test Setup
The mesh test setup for the HomeKit add-on is shown below. This setup is similar to the wired test setup and requires the use of a satellite/mesh node which has a wireless backhaul connection to the primary AP. This setup is used to verify Wifi to WiFi client behavior through a satellite/mesh node specifically.
CDRouter’s first LAN interface is configured to connect to the primary AP via WiFi by specifying its BSSID (BSSID1). A second WiFi LAN interface is enabled and configured within CDRouter to connect to the satellite/mesh node by specifying its BSSID (BSSID2).
Test Coverage Table
The Apple HomeKit Secure Router Certification Test Plan currently defines 53 test cases which range from fully automated to semi-automated to fully manual.
|Test Type||Number of Tests||Requires User Intervention?||Included with CDRouter?|
All semi-automated and manual tests require some level of manual intervention such as requesting the user to perform some specific action in order to continue or complete the test. Fully manual tests cannot be performed by CDRouter and are not included with the HomeKit add-on.
The fully manual tests not included with CDRouter are:
All semi-automated and automated tests can be performed by CDRouter and are included with the add-on. Semi-automated tests will require user intervention and will pause and provide instructions to the user as needed.
The semi-automated tests included with CDRouter are:
To enable uninterrupted testing of the fully automated tests only, all semi-automated tests can be skipped by setting the testvar homeKitManualEnabled to a value of no.
Some fully automated tests require additional configuration. Specifically, tests TCSR007 and TCSR039 require the use of a satellite node (these should only be run if the DUT supports mesh functionality) and test TCSR006 requires a guest network.
This testvar enables HomeKit functionality with in CDRouter. When set to a value of yes CDRouter will enable HAP on all LAN clients created during start, and automatically communicate with the computer running HAT to control the DUT.
testvar supportsHomeKit yes
Configuring the DUT
This testvar must be set to the name of the HomeKit-enabled router or DUT as seen in HAT. HAT will treat this device as the DUT during all HomeKit tests.
testvar homeKitRouterName myRouter
This testvar must be set to the name of the machine on CDRouter’s LAN test network which is running HAT. This machine is then discovered in start using mDNS.
testvar homeKitHATHostname homekits-MacBook-Pro.local
Configuring the Rules Files Path
This testvar must be set to the absolute path of the directory on the computer running HAT where all JSON rule files are stored.
testvar homeKitHATRulesPath /path/to/rules/
Configuring the Setup Code for Pairing
This testvar must be set to the setup payload used to pair with the DUT. This value is used by CDRouter in tests that verify the behavior of pairing.
testvar homeKitRouterSetupCode "X-HM://00WP2P2SLNJ15"
Enabling Semi-automated Tests
There are numerous tests in the homekit test module that require some degree of user intervention. These tests can all be skipped easily by setting the testvar homeKitManualEnabled testvar to a value of no. By default this testvar is set to yes meaning all tests in the homekit module will run. Any tests that require user intervention will pause and present a dialog box indicating what action is required by the user in order to continue.
A common use case for wanting to skip the manual tests is to enable uninterrupted overnight runs of the homekit module.
testvar homeKitManualEnabled yes
CDRouter’s LAN client rotation feature should be disabled when running HomeKit tests. This can be accomplished by setting the useSameLanInterface testvar to a value of yes.
testvar useSameLanInterface yes