Internet of Things Network Study

By Masters in Embedded and Cyber-Physical Systems

Research Advisor Dan Cregg Team members Ta-Yu Chiu Neha Sadhvi Yidi Fan Ricky Yang

Contents

1.Thesis Statement 3

2. Introduction 3

3. Objectives 3

4. Equipment and Devices 3

5. Network Protocols in Study 4

6. Testing Environment 6

7. Number of devices under Test 6

8. Network Layer Frequency 6

9. Physical Layer Frequency 6

10. Test Setup 7

11. Results 8

12. Constraints 13

13. Conclusion 14

14. References 14 15. Acknowledgements 15

16. Contacts 15

1.Thesis Statement As the use of smart embedded devices grows in our daily life, current networking technologies are expected to be strained beyond their original intent. Consumers face unacceptable performance as nodes are increased and network bandwidth is consumed in physically constraining environments. Various network types and use cases, thus, are explored to determine current failure points in common IoT home smart devices. 2. Introduction At the time of this report, there has been an unprecedented uptake in the use of ‘smart’ devices in the home. The introduction of voice recognition platforms such as Google Home and Amazon Alexa has fueled the use of small, inexpensive, connected sensing and control devices. Controls for Heating and Air Conditioning have been popular, as well as various sensors for doors, windows, etc.. Perhaps the most pervasive has been the acceptance of these types of systems for lighting control. Due to the sheer number of nodes available in a home, lighting nodes to be controlled is very large. This research will focus on various types of network technologies with the goal of physically simulating small to large sets of devices to determine acceptable response time. 3. Objectives In order to determine acceptable consumer performance levels and evaluate selected IoT network protocol, we severely tested the 4 well-known network protocols under various real environmental conditions. We also tested these protocols for its scalability by connecting at least 40 devices to the same network to study its performance. To best simulate a typical use case, it is expected that the devices would not all be expected to be at the exact same state (light level for example) and therefore, direct or individual commands were used as required. 4. Equipment and Devices ❏ ESP-32 WiFi / module ❏ 2457D Plug-in Dimmer Module ❏ Insteon Hub ❏ Insteon Powerline Filter ❏ Philips Dimmable LED bulb ❏ 120VAC-5VDC USB power supplies ❏ STE3000 RF Test Shielded Enclosure ❏ Philips Hue connect LED bulb ❏ Philips Hue Bridge

Equipment used for Network Protocol Testing

5. Network Protocols in Study ● Bluetooth : Bluetooth is a wireless technology for wirelessly exchanging data over short distances using ultra-high-frequency radio waves. Technically, Bluetooth operates in the 2400-2483.5 MHz range within the ISM 2.4GHz frequency band [1]. This technology was designed to eradicate the use of chords, cables etc. when the devices are in close proximity to each other. However, Bluetooth was originally designed for continuous, streaming data applications; in other words, it is better to apply this technology in the audio, or video transmitting. Thus, the applications of Bluetooth technology in IoT mostly tend to be like a wireless speaker, files transfer between devices, or wireless headsets. On the other side, BLE (also known as ) is more widely used in the domain of IoT. Thus, we adopt BLE for our testing. For BLE, it also operates the same frequency band as Bluetooth. The reason why we use BLE is that in most cases, smart home devices are mostly dealing with the very simple messages like just turn on/off of appliances, or discrete messages. All of these operations are very simple for not only users but also devices. For instance, if a user wants to operate his/her air conditioner, he/she just needs to issue a message, like “ON” signal to the device, from his phone, and the device just needed to make sure it can receive the message and turn itself to ON status. Furthermore, BLE remains in sleep mode constantly except for when a connection is initiated [2], which can largely decrease the power consumption if we are compared with classic Bluetooth. ● Insteon : With the advent of the IoT industry, lots of wireless protocols are being used in this field and each one is trying their best to make themselves more attractive in this large and competitive market. Insteon, which is also a very popular wireless protocol for IoT, is started in 2005. With years and years of efforts on improvements and adjustment, Insteon protocol is becoming a very

powerful and effective wireless protocol. For its technology, Insteon adopts a dual-band technology in which it uses both powerlines as well as wireless technology which allows the signal to travel further without interruption, resulting in a lower effective data loss [3]. Moreover, the use of a simulcast mesh network allows better scalability to adapt to a future of automated homes. To be more concise, every device can repeat messages across the network, eliminating the bottlenecks that occur when a single device fails in a routed network [3]. Also, because of dual-band technology, the mesh network extends over both wireless and wired Insteon devices, which can dramatically lower the data miss rate.

● Wi-Fi : Wi-Fi is a popular wireless networking technology which could provide high-speed internet and network connections using radio waves. It’s frequently used for the wireless local area networking (WLAN) of devices which is based on the IEEE 802.11 family of standards. [4]Wi- Fi technologies have a broad implementations on devices including desktops, laptops, smartphones and smart home devices. Compatible devices can connect to each other over Wi-Fi through a wireless access point as well as to connected devices and may use it to access the Internet. Such an access point has a range of about 20 meters (66 feet) indoors and a greater range outdoors. [5]For instance, the coverage of Wi-Fi can be as small as a single room with walls that block radio waves, or as large as many square kilometres achieved by using multiple overlapping access points. Nevertheless, Wi-Fi is potentially more vulnerable to attack than wired networks because anyone within range of a network with a wireless network interface controller can attempt access. To overcome the problem, Wi-Fi Protected Access (WPA) is established to protect information moving across Wi-Fi networks and includes solutions for personal and enterprise networks. Security features of WPA have included stronger protections and new security practices as the security landscape has changed over time. ● ZigBee : Zigbee is an IEEE 802.15.4-based specification for a suite of high-level communication protocols used to create personal area networks with small, low-power digital radios, such as for , medical device data collection, and other low-power low-bandwidth needs, designed for small scale projects which need wireless connection. Hence, Zigbee is a low-power, low data rate, and close proximity wireless ad hoc network. The technology defined by the Zigbee specification is intended to be simpler and less expensive than other wireless personal area networks (WPANs), such as Bluetooth or more general wireless networking such as Wi-Fi. Applications include wireless light switches, home energy monitors, traffic management systems, and other consumer and industrial equipment that requires short- range low-rate wireless data transfer.

6. Testing Environment

● Open Area Scenario: No obstacle in between the devices; in other words, all devices are placed in the same room along with the controller.

● Common Residential Scenario: To simulate a common house with a wooden wall, we kept all the 40 devices in the same closed room and kept the controller in the different room to test the penetration capacity of the protocols through the walls of the room. ● Concrete Material Scenario: To simulate a scenario where the signal penetrates through the concrete material and measure each protocol’s performance. We kept all of the 40 devices on the second floor and used the controller from the first floor to test the performance of each protocol via the concrete floor material. ● Metal Material Scenario: Server devices at placed inside a metal box to test each protocol under the most extreme conditions. This scenario is not likely to happen in real life, but it is a good measurement to measure each protocol in the worst possible conditions. ● Real Home Environment Scenario: Controller and all the 40 devices are scattered around different places inside the real home to test each protocol’s performance in the presence of all of the above-mentioned materials i.e. wood, concrete, metal and many other materials generally available in the house. 7. Number of devices under Test ● Bluetooth: 40 ● Insteon: 40 ● Wi-Fi: 40 ● Zigbee: 40

8. Network Layer Frequency ● Bluetooth: 2.4 GHz ● Insteon: 915 MHz ● Wifi: 2.4 GHz ● Zigbee: 2.4 GHz

9. Physical Layer Frequency ● Bluetooth: None ● Insteon: 132KHz Powerline ● Wifi: None ● Zigbee: None

10. Test Setup ● Bluetooth: For the Bluetooth devices, there can be various different topology set up that could be used, like point-to-point, broadcast and mesh network topology. For our testing, we selected the

point-to-point network topology to best simulate a home use-case. We used 40 ESP32 microcontrollers having a BLE module as clients to receive the data of turning on the onboard LED from the server(controller) which is again an ESP32 module. The server sends the data to turn on the light one by one to each of the 40 devices. For this, in our project, we used C++ programming language for the development work and used Arduino as our Integrated Development Environment (IDE) to write and embed code into each of our 40 devices and one server. Then we used these devices to test for their performance in all of the 5 aforementioned environments. ● Insteon: For Insteon, we took 40 Insteon 2457D plug-in dimmer modules and paired them all to the single Controller which was again an Insteon 2457D plug-in dimmer module. We attached 40 Philips Dimmable LED bulb to each of the 40 lamp link modules. The controller sends the message to turn on and off the lights to all of these 40 devices at the same time. We then note down the delay and data loss that we saw during our testing of these modules in these 5 given environmental conditions. ● WiFi: 40 ESP-32 microcontrollers are used as client devices. A MacBook laptop is used as a server (controller). For each testing environment, routers and client/server devices are placed so that the signal has to go through different obstacles. Each client device is programmed so that its onboard LED indicates its connection status - disconnected, connected and waiting for the signal, the signal received. A python script is executed on the server to send the signal to each client. Once the client receives the signal, it will respond with a timestamp to measure the delay. Data loss is measured by comparing the number of devices connected to the network to the number of devices responded. ● Zigbee: 40 Philips Hue light bulbs utilizing Zigbee and a Philips Hue Bridge were used for Zigbee testing. Once a light bulb is connected to the network, it will show up on the Philips app as controllable. The data loss is measured by the response from the light bulbs, and the delay is measured by examining a video recorded during the test. The same setup is used for all five environments to measure its performance under different scenarios. 11. Results We tested all the four protocols rigorously and compared all of them to find which protocol performs better when exposed to the same scenario where we have a huge number of devices connected to the same network. Below are the pictures and graphs attached where we can see the performance results of these protocols under various environmental conditions.

[1] Results of Zigbee in Open Area with devices and controller in: (a) same room (b) separated by concrete wall

[2] WiFi Controller inside the Metal Box and 40 devices outside

[3] Bluetooth devices and controller in rooms separated by Common Residential Material

[4] Insteon devices in different rooms in Real Home Environment

[5] Open Area Comparison Graph

[6] Common Residential Area Comparison Graph

[7] Concrete Material Comparison Graph

In the scenario of Open Area, Common Residential Area, and Concrete Material Area graph, we can see that the performance of Insteon and Zigbee is very similar to each other. But Insteon still outperformed Zigbee in terms of connection speed and data loss, with Wifi following closely behind. On the other hand, we can see that Bluetooth devices took the longest to connect to the network and are easily affected by obstacle between line of path. We can see that, for Insteon, all the 40 devices got connected in all the above scenarios within 1 to 2 seconds. With the Zigbee, we found that approximately 35 devices gets connected within 1 second but other 4 to 5 devices were not able to receive the data from the controller. In case of WiFi, we found that all of the devices were able to receive the data from the controller but the time it took for all the devices to receive the signal from the controller is about 5 seconds which is far more than what Insteon and ZigBee devices took to get connected. Whereas in case of Bluetooth, we see huge amount of data loss and well as it took a very long time, in comparison to other protocols, for the devices to receive the data from the controller. We can see from the above results that the bluetooth gets heavily affected and the signal gets largely attenuated in the presence of various obstacles.

[9] Metal Box Comparison Graph

In the result of Metal Box graph shown above, we can see the performance of Zigbee, WiFi and BLE are not very well - a significant amount of devices failed to connect to the network to receive the data. The Metal Box we used can block 90% of the Radio Frequency (RF), which is heavily relied on by these three protocols. On the other hand, Insteon still shows a consistent performance because its powerline layer is not affected by the metal box. So, with the graph we can see that approximately all the 40 devices got connected and were able to receive the message within 1 to 2 second and there were no data loss captured with the Insteon devices which was mostly because even when we blocked most of the RF signal, Insteon controller device was able to transmit the data via the powerline. For Wi-Fi, there was a significant degradation for its performance in the metal box test. From the graph, it is shown that the devices started to connect to the network 7 second later. Also, some of the devices will not be able to connect after 3 minutes and there appears to have some data loss. In this case, the performance of Wi-Fi could be influenced by the interference of metal box. For Zigbee, there are some devices connected in the beginning second but the number of devices connected has been decreased in comparison to the other tests. Also, some devices were not able to receive the message and there was a significant data loss. For the Bluetooth, we see that at the maximum 2 to 3 devices were only able to receive the data and rest of the other 37 devices were not at all able to receive the data from the controller when the RF signal was blocked with the metal material. And, also Bluetooth devices took approximately 9 to 10 seconds to receive the signal which is far more in terms of time when compared to other protocols.

[9] Real Home Scenario Comparison Graph

Based on the result graphs shown above, we can see that the four protocols perform very differently in the Real Home setting, where all devices are scattered around in the house with the presence of the mixture of various different materials like wood, concrete, glass, metals etc., which can hinder the transmission of data from the controller to the IoT devices. In this kind of setting, Bluetooth shows the worst performance with significant data loss as it gets severely affected by the presence of these materials. 12. Constraints

● Bluetooth: One of the constraints we found with bluetooth devices in our test is that the BLE devices gets easily affected by the obstacles in between the controller and the devices that needs to be controlled. In other words, if there is an obstacle between the controller and the devices, the percentage of messages being received by the devices will heavily reduce. Also, during our testing, if we increase the distance between masters and clients, the data lose will then significantly increase. However, if BLE devices are all in the same place, like an open area where there are fewer obstacles present, the success rate of signal propagation is increases.

● Insteon: With Insteon, we did not find any constraint which can adversely affect our testing. As Insteon supports two physical layers, which is, Radio Frequency (RF) and Powerline, therefore, even in the situations where RF does not work well, Insteon manages to work pretty well because of the presence of powerline transmission protocol.

If we see other wireless protocols, the performance reduces drastically in the situations where RF signals gets affected due to the presence of the various different materials. Because Insteon protocol can transmit their message through powerline, it still performs well in most of the environment and in the presence of most of the materials without any visible data loss.

● Wifi: Overall, Wifi performs well in most situations except the metal box testing. In a real home environment, all WiFi devices get connected in approximately 6 seconds and they were able to receive the message sent from the router. However, in the metal box testing environment, there exists a significant performance degradation and it turns out that WiFi is vulnerable to signal interference. From the resulting graph, it also established that WiFi devices are not connected to the network immediately and there will be a delay for 1-2 seconds. Comparing to Insteon and Zigbee, Wifi requires more overhead since each device has to occupy a unique IP address, even if the data is transmitted infrequently between those devices.

● Zigbee: From the result of testing, Zigbee devices established a fast reaction to the message sent from the hub. However, its performance suffers as the number of devices in the network increases. It has better connection speed compared to Wifi, but suffers from devices failing to connect. Its overall performance is also lacking compared to Insteon in all five scenarios in terms of data loss and delay. 13. Conclusion As consumers continue to discover new use cases for embedded connected devices in their homes, the number of nodes will grow dramatically. With this growth, the challenges with existing network protocols will create unacceptable performance levels in many use cases. Reliability and scalability will be key performance factors for continued adoption. Use of redundant physical communications and scalability without latency will be required. This study finds that the protocols that support multiple physical paths for redundancy, as well as scalability to large numbers without creating visible latency issues for the consumers, are the best approach for future growth.

14. References [1] Ray, B. (n.d.). Bluetooth Vs. Bluetooth Low Energy: What's The Difference? Retrieved from https://www.link-labs.com/blog/bluetooth-vs-bluetooth-low-energy [2] Topology Options. (n.d.). Retrieved from https://www.bluetooth.com/bluetooth- technology/topology-options/

[3] Technology. (n.d.). Retrieved from https://www.insteon.com/technology#ourtechnology [4] Garber, Megan (2014-06-23). "'Why-Fi' or 'Wiffy'? How Americans Pronounce Common Tech Terms". The Atlantic. Archived from the original on 2018-06-15. [5] Beal, Vangie. "What is Wi-Fi (IEEE 802.11x)? A Webopedia Definition". Webopedia. Archived from the original on 2012-03-08. [6] Ray, B. (n.d.). The ZigBee Vs WiFi Battle For M2M Communication. Retrieved from https://www.link-labs.com/blog/zigbee-vs-wifi-802-11ah [7]Alliance, Z. (n.d.). Retrieved from https://web.archive.org/web/20130627172453/http://www.zigbee.org/Specifications/ZigBee/FA Q.aspx

15. Acknowledgements We would like to express our sincere gratitude to several individuals and organizations for supporting us throughout our Master study. First and foremost, we wish to express our sincere thanks to our supervisor, Dan Cregg, for his patience, enthusiasm, insightful comments, invaluable suggestions, helpful information, practical advice and unceasing ideas which have helped us tremendously at all times during our research. His immense knowledge, profound experience and professional expertise in IoT has enabled us to complete this research successfully. Without his guidance and relentless help, this research would not have been possible. We could not have imagined having a better supervisor in our research work. Also, our sincere thanks go to Professor Eltawil, UCI, for his continuous support and assistance in our graduate study. We are also grateful to Robert and Blake, Engineers at Insteon, for their kindness, hospitality and technical support. In addition, I would like to acknowledge the University of California, Irvine, for organizing various courses, which have helped us in improving our research and programming skills.

16. Contacts ● Ruiqi Yang: [email protected] ● Yidi Fan: [email protected] ● Neha Sadhvi: [email protected] ● TaYu Chiu: [email protected]