Wireless Communication Testbed and Tools for Authentic STEM Learning

Paper ID #13504

Vuk Marojevic graduated from University of Hannover (MS), Germany, and Polytechnic University of Catalonia (PhD), Spain, both in electrical engineering. He joined Wireless@Virginia Tech in Summer 2013 as a Research Associate. His research interests are in software-deﬁned radio technologies with application to 4G/5G cellular, UAV, and spectrum sharing, among others. Dr. Marojevic has been instructor of undergraduate and graduate level classes at the Polytechnic University of Catalonia, Spain, and Virginia Tech. Dr. Richard M. Goff, Virginia Tech Department of Engineering Education

Richard M. Goff is a former aircraft structural test engineer for the Navy, Peace Corps Volunteer, and computer entrepreneur; he holds a Ph.D. in Aerospace Engineering, and is currently an Associate Professor in the Department of Engineering Education at Virginia Tech. Richard has been teaching and engaging in research in multidisciplinary engineering design education for over eighteen years. Dr. Goff is the recipient of several university teaching awards, outreach awards, and best paper awards. His passion is creating engaging learning environments by bringing useful research results and industry practices into the classroom as well as using design research results to inform engineering practice. Dr. Carl B Dietrich P.E., Bradley Department of Electrical and Computer Engineering, Virginia Tech

A licensed Professional Engineer in Virginia, Carl Dietrich earned a BS EE degree from Texas A&M University, and MS EE and PhD EE degrees from Virginia Tech. He has taught courses in software defined radio, communications systems, electronics, and electromagnetic fields. He has also taught short courses on software defined radio since 2007, covering fundamental concepts and enabling technologies in addition to the use of open source software to develop and run SDR applications. In addition, Dr. Dietrich has performed and directed research in the areas of cognitive radio, software defined radio (SDR), multi-antenna systems, and radio wave propagation, and has authored or co-authored more than 50 peer- reviewed journal and conference papers. He has worked at Virginia Tech, Bell Northern Research, and the Defense Information Systems Agency. He has served as chair of the Wireless Innovation Forum’s Educational Special Interest Group, is a member of ASEE and Eta Kappa Nu, Senior Member of IEEE, and an Extra class amateur radio operator. Dr. Taeyoung Yang, Virginia Tech

Taeyoung Yang received the M.S. and Ph.D. degrees in electrical engineering from Virginia Tech in 2003 and 2012, respectively. He received B.S. and M.S. degrees in electronic engineering from Sung-Kyun- Kwan University, South Korea, in 1997 and 1999, respectively. He is currently a Research Scientist with the Bradley Department of Electrical and Computer Engineering at the Virginia Tech. Dr. Yang has strong theoretical background on radiation physics, co-site interference, and size-performance limits of wireless devices. Along with the theoretical background, he also has extensive hands-on experience in antennas, RF, sensors, and measurements. His current research emphasis is on developing core technologies including generalized fundamental limit theory on antenna size and performance, spectrum sharing and management, cognitive wireless infrastructure, software-deﬁned radio user equipment, and mission- critical cloud services. Dr. Christian W. Hearn, Weber State University

Dr. Hearn is an Assistant Professor in the College of Applied Science and Technology at Weber State University. He received a Ph.D. in Electrical Engineering from Virginia Tech. He was a member of the Virginia Tech Antenna Group during his plan of study. Before returning to graduate school, Mr. Hearn was a mechanical engineer for the Naval Surface Warfare Center. He is a licensed mechanical engineer in the state of Virginia.

c American Society for Engineering Education, 2015 Paper ID #13504

Dr. Nicholas F Polys, Virginia Tech Advanced Research Computing

Nicholas F. Polys is Director of Visual Computing with Virginia Tech Research Computing Group and Afﬁliate Research Professor in the Department of Computer Science. He has developed interactive 3D graphic content and systems for over 15 years. His research interests lie at the center of graphics and Hu- man Computer Interaction: the intersection of visualization, virtual environments, and perception. After his undergraduate research in Cognitive Science at Vassar College (1996), he jumped into the networked information space of the WWW developing audio, visual, and 3D assets and software. His doctoral work at Virginia Tech (2006) examined perceptual cues and layout behaviors for Information-Rich Virtual En- vironments for desktop to immersive platforms. He is a member of ACM, IEEE Computer Society, and the Web3D Consortium. He is a co-author of the international standard (ISO) Extensible 3D (X3D), elected Director and President of the Web3D Consor- tium, and Chair of the Web3D User Interface Working Group. Prof. R. Michael Buehrer, Virginia Tech

Dr. R. Michael Buehrer joined Virginia Tech from Bell Labs as an Assistant Professor with the Bradley Department of Electrical and Computer Engineering in 2001. He is currently a Professor of Electrical Engineering and is the director of Wireless @ Virginia Tech, a comprehensive research group focusing on wireless communications. During 2009 Dr. Buehrer was a visiting researcher at the Laboratory for Telecommunication Sciences (LTS) a federal research lab which focuses on telecommunication challenges for national defense. While at LTS, his research focus was in the area of cognitive radio with a particular emphasis on statistical learning techniques. His current research interests include geolocation, position location networks, iterative receiver design, dynamic spectrum sharing, cognitive radio, communication theory, Multiple Input Multiple Output (MIMO) communications, intelligent antenna techniques, Ultra Wideband, spread spectrum, interference avoid- ance, and propagation modeling. His work has been funded by the National Science Foundation, the Defense Advanced Research Projects Agency, Ofﬁce of Naval Research, and several industrial sponsors. Dr. Buehrer has authored or co-authored over 50 journal and approximately 125 conference papers and holds 11 patents in the area of wireless communications. In 2010 he was co-recipient of the Fred W. Eller- sick MILCOM Award for the best paper in the unclassiﬁed technical program. He is currently a Senior Member of IEEE, and an Associate Editor for IEEE Transactions on Communications and IEEE Wireless Communications Letters. He was formerly an associate editor for IEEE Transactions on Vehicular Tech- nologies, IEEE Transactions on Wireless Communications, IEEE Transactions on Signal Processing, and IEEE Transactions on Education. In 2003 he was named Outstanding New Assistant Professor by the Vir- ginia Tech College of Engineering and in 2014 Dr. Buehrer won the Virginia Tech College of Engineering Award for Teaching Excellence.

c American Society for Engineering Education, 2015 Wireless Communication Testbed and Tools for Authentic STEM Learning

Abstract We propose a novel educational gamification approach that employs and reinforces otherwise- abstract concepts currently taught in graduate-level courses to become a standard part of undergraduate communications courses in the future. In particular, we develop software tools that vis- ually demonstrate relevant wireless communications parameters and processes, including those already taught in undergraduate communications courses (e.g., power, coding rate, modulation type, data rate and bandwidth) to students in an interactive way. We exploit our Internet- accessible wireless communication testbed CORNET to enable students to experience wireless communication challenges and learn different solutions in realistic environments via an ordinary web browser. This paper presents our approach to authentic STEM learning and compares it with other past and ongoing initiatives. We conclude that there is a gap of practical teaching tools and methods for educating students as well as radio engineers and researchers about new trends in wireless communications with a focus on spectrum sharing.

1 Introduction

Wireless communications research is investigating how to make better use of the radio frequency (RF) spectrum. Increasing the throughput per unit bandwidth (spectral efficiency) is a popular choice that is continuously improved and employed in practice. Reusing frequency bands oppor- tunistically (dynamic spectrum access or DSA) is still in the research phase, but is likely to make its transition to practice in the near future based on the currently ongoing license auctions for the AWS-3 and other bands in the United States.

These techniques are covered in graduate-level electrical engineering classes, but cannot easily be introduced to undergraduate students. Undergraduate engineering students struggle with the basic concepts in wireless communications. We therefore propose the development of visualization and gamification tools to be used in undergraduate courses. Gamification refers to the use of game design elements in non-game contexts, products, and services1,2. Gamification is also considered a useful tool to motivate and enhance learning. To apply gamification to educate about basic and modern wireless communications principles, a wireless communications simulator or testbed is needed. A testbed allows for actually transmitting and receiving data over the air.

A cognitive radio testbed consists of cognitive radio nodes. Each node has a cognitive agent or engine that controls a flexible radio or set of radios. The cognitive engine is usually implemented as a complex algorithm that may evolve over time. Initially, an operator or agent may have very limited knowledge about how to effectively operate the radios. A cognitive agent is able to learn how to adapt its information transmission and other radio parameters to a given situation. Our approach is to replace the cognitive agent with a student. A student can then control a flexible radio in a wireless testbed under a gaming scenario to achieve a challenging wireless communication “mission”. Multiple students can participate in the scenario as individuals or in groups. As students operate their radios, they will observe cause and effect and learn to apply and improve their knowledge of material currently covered in undergraduate communications courses in order to make better decisions and achieve more rewarding outcomes. Students will be in situations where there is a need to cooperate their transmissions with their classmates to get the best overall benefit (score) or operate selfishly to maximize their own score. Lessons learned through this game-like process representing real-world communication challenges can be further discussed in class. We expect that students will be strongly motivated to learn more about advanced wireless communication principles and to learn how cognitive radios work.

Figure 1a illustrates a cognitive engine (artificial intelligence to provide an optimized communication performance) taking as inputs the wireless environmental parameters to turn the knobs and meters of the radio or radios it controls based on the acquired knowledge and policies. The cognitive engine in Figure 1b is the student, who observes and controls the radio(s) and learns from their own and other students’ actions.

(a) (b) Fig. 1. Cognitive radio system (a) and gamification concept for education (b).

This paper will present a prototype of a wireless testbed using the educational gaming approach. We will discuss example scenarios, visualization tools, class modules, and evaluation methods. Evidence from past experience using similar methods will be also discussed. We believe that the learning effect and engagement will significantly improve learning of undergraduate communications engineering content over traditional methods and also promote and enable introduction of advanced wireless communication concepts into the undergraduate curriculum. To ensure wide- spread relevance of the tutorials, we are developing them in collaboration with faculty from a variety of institutions and programs that represent undergraduate-only engineering and engineering technology programs, historically black colleges and universities, and research universities. We hope that the proposed gamification approach can be effectively introduced into other cur- ricula to enhance undergraduate and graduate education and research engagement in related engineering and science fields.

2 Problem statement and objectives

There is a jumble of symbols, concepts, channels, systems and standards that make wireless communications sound very abstract and incomprehensive. (This is true for other engineering and science fields, too, but we focus on wireless here.) The question we raise is can we provide better access and understanding of these concepts and introduce them into the undergraduate engineering curriculum?

Rather than looking at equations and variables, our objective is providing an intuitive and practical understanding of wireless communications principles to a student population that has limited theoretical background. We propose an interactive human-computer interface (HCI) that provides users with human-intelligible feedback on the radio environment as a function of time, frequency and space to leverage active learning by letting the students become the operators of a wireless communications system. This will provide a better feeling for the parameters that enable as well as govern real-time information exchange over-the-air, making them more tangible and intuitive like a steering wheel, gear, shift, and pedals are for driving a car.

This paper presents our initial work on developing such educational tools using our Cognitive Radio Network Testbed (CORNET). The tools will be part of a new teaching methodology that complements traditional learning methods and can be applied before, during, and after describing the theory. Our long-term objective is to examine the following educational research questions: 1. Does a visual lab learning environment make the field of radio communications more accessible to undergraduate students? 2. Is learning improved when compared to traditional teaching methods? 3. Do these undergraduate students persist in the field? 4. Are the students more motivated to learn and feel integrated in the learning process independent of their background and experience?

The long-term effect will be demonstrated in terms of professional success and an evolved society.

3 Background and related work

Successful university-level STEM programs continuously adapt faculty expertise and improve undergraduate curriculums to advances in technology. One example addressed by CORNET is dynamic spectrum access (DSA). DSA, also referred to as dynamic spectrum management (DSM), is a set of techniques based on concepts in network information theory and game theory utilized to create an adaptive spectrum allocation model for wireless communications. DSA has the potential to overcome impending interference limitations and significantly improve the capacity of high-volume communication networks.

Historically, interference in wireless communications has been mitigated with the regulation of frequency spectrum at the government level. Static spectrum management has effectively mini- mized interference in wireless communications. However, it can result in inefficient radio spectrum use and has become a limitation with increasing demand for broadband wireless internet access. Suitable radio spectrum (the portion with the best trade-off between transmission range and available bandwidth, between 300 MHz and 3 GHz) has become a valuable resource. Worldwide studies also indicate these bands are underutilized within the static spectrum management model3. As a result, researchers and spectrum regulators have proposed DSM to optimize spectrum use.

DSA systems can adapt carrier frequencies and bandwidth of transmitted signals to the operating environment. Unused spectrum bands may be utilized in an adaptive manner to optimize the use of radio spectrum. The term cognitive radio may also be used to describe wireless systems capable of DSA. A cognitive radio autonomously decides how to access the spectrum; a cognitive engine processes environmental information and learns how to turn the knobs of the radio transmitter, receiver or system. When the cognitive radio is based on software-defined radio (SDR) technology, it has a high degree of flexibility to implement the waveform that best adapts to the current radio environment and internal conditions, such as available processing resources and battery capacity. Cognitive radio can enhance many communications principles, including link adaptation, bandwidth management, channel access, multiple-input-multiple output (MIMO) communication, pre-cancellation of estimated interference.

Gamification is the application of digital game techniques to ‘non-game’ problems4. Current trends in education research indicate education-based games motivate and engage users, rein- force rational reasoning, and facilitate learning. Interest in innovative methods for human- computer interaction and cognitive psychology can be leveraged to develop educational tools for teaching advanced engineering principles.

Related work in the field of gamification for wireless communications include the MANIAC challenge for ad-hoc networks, the DARPA spectrum challenge for agile interference-tolerant wireless systems, the Engineering Education Island project developed at the University of Ulster, and an interactive DSA game utilizing software radios called Spectrum-wars.

The MANIAC Challenge5 is a competition created to better understand cooperation and interop- erability in ad-hoc networks. Student teams comprise a wireless ad-hoc network. Organizers generate traffic destined to some point in the network. A hop-by-hop bidding contest decides the path of each data packet to its destination. Teams are judged by the percentage of relayed traffic that reaches its destination.

In 2013, the Defense Advanced Research Projects Agency (DARPA) held the first DARPA Spectrum Challenge6. Participants in the challenge competed to demonstrate an optimal radio protocol for a given communication channel in the presence of other dynamic users and interfer- ing signals. Teams integrated their own software radio with common radio hardware to evaluate link performance in a structured test-bed environment.

The Serious Games and Virtual Worlds research team at the University of Ulster-Intelligent Sys- tems Research Center created the Engineering Island project to investigate how virtual worlds (VWs) could be effectively used for teaching and learning. The VW environments contain a range of interactive engineering demonstrations that allowed the project to assess the perceived benefits of three-dimensional VWs for education7.

Spectrum Wars borrowed from both MANIAC and DARPA spectrum challenges. However, where both the MANIAC and DARPA projects required dedicated communication systems and user expertise, Spectrum Wars is educational tool in a game format that can be played by participants with no prior knowledge of wireless communication systems in general8. 4 Approach

We leverage our wireless testbed, combining different 2D and 3D views of the radio environment with gamification features. Below we present the testbed, the design choices and tools.

4.1 CORNET

CORNET mimics a real-world wireless system while providing a safe operating environment for over-the-air transmission at low power or under experimental licenses9. It consists of 48 SDR nodes that are spread on four floors of a building (Fig. 2). A node can be individually or jointly accessed to develop, execute, or control a radio transmitter, receiver or network. CORNET is unique among national and international tesbeds of its kind10. It offers free access to a wide range of experimental research and educational tools based on open-source software and flexible hardware. An FCC experimental license agreement allows using several frequency bands for testing new wireless communications techniques and technologies.

One of the primary concepts behind cognitive radios is the ability to dynamically adapt the radio transmission parameters, including the transmission band. The second generation of Universal Software Radio Peripheral, USRP2, provides this capability by using tunable RF daughter boards11. Each SDR node thus consists of a USRP2 radio front ends with WBX daughterboard capable of tuning to center frequencies between 50 MHz to 2.2 GHz. Each USRP is connected via Gigabit Ethernet to a powerful server which runs the digital signal processing algorithms that define the shapes, the signal, and defines the channel access mechanism. A gateway server provides remote access to each one of the radio nodes.

Our testbed provides an ideal environment for developing, testing and evolving visualization and gamification tools that illustrate current challenges in wireless communication to undergraduate engineering students.

Fig. 2. The CORNET testbed floor plan with the 48 radio front ends as an HTML5 X3DOM data-driven visualization.

4.2 Design choices

4.2.1 Visualization

Learning fundamental and advanced radio resource management concepts needs a prior understanding of many concepts. Some of these can be modeled, making them tangible for undergraduate students. Visualization is a powerful tool as it allows hiding or highlighted specific aspects or effects of the radio environment. We expect that the cognitive scaffolding approach that we have developed in a previous project will be useful in the early stages of design. Specifically, it can serve as an analytic evaluation tool to guide early versions of the software. However, as the iterations progresses, the value of empirical evaluations increases. This dynamism makes sense as the development effort shifts from the fundamental functionality to the usability of the client interface in supporting the specific activities and detailed work flows of the game play and the undergraduate assignments.

In order for the broadest impact and adoption of our curriculum modules, we will build the plat- form with web-based technology. The user interface then, is simply an HTML5 web page that can be loaded on desktops and mobile devices. We will build the 3-D Game based on the robust open-source X3D WebGL library, X3DOM.org12,13. X3DOM enables interactive, hardware- accelerated 3-D graphics natively in a web-page, with the ability to share events and data with the ecology of web media and services. Multi-user, distributed virtual environments have recent- ly been demonstrated using only ‘native’ web technologies14,15.

 Color—Color perception is a fundamental example of pre-attentive processes and scientific visualization practice still lagging well behind research16,17. In addition, we must also consider other attributes that impact a user's perception, interpretation and sense- making18. Specifically, transparency, iso-surface contours, and glyphs can all be used to represent different quantitative variables over time and space.

Tradeoffs will need to be made to limit the communications and processing delay for real-time experience. The colors and number of involved colors or shades need to be carefully chosen. Also, only the relevant information should be displayed to catch the student’s focus on a particular aspect of the environment. We have faced these tradeoffs with our initial visualization implementation.

 Human-computer interface (HCI)—We propose to enable humans to interact with the radio frequency signal environment through an interactive user interface that engages human perception and information processing capabilities. This interface will allow spectral opportunities to be perceived, creating spectral glasses for the eyes (sensors) and the brains (humans or cognitive engines).

4.2.2 Gamification

Previous educational research concluded that the games need to have a special meaning to the student, inspire and provide virtually unlimited choices1.  Competition versus cooperation—Competition and cooperation can both increase motivation. A student who is motivated to learn often experiences the greatest reward when being able to help other students master the subject, too. Both techniques will therefore be considered. That is, some lab sessions will be of competitive and others of collaborative nature.

 Switch among different representations—Gilbert et al.19 points out that there is no rule for the instructor to know when to switch between representations. We believe that a gradual switch from simple to sophisticated or complex, multi-resource abstractions will help the students understand complex processes. Alternatively, diving in and out and presenting the subject from different angles can be well suited in certain situations and as a function of the students’ capability to follow such changes. The different representations should complement each other and provide a toolbox for students to engage in active learning. Understanding the duality between traditional, textbook representations and new ways of presenting the physical world is essential and needs to be emphasized. Op- tions to highlight specific features and hide other will help the students understand the different information contained in the different representations and help them understand which representation to choose or filters to apply for analyzing a given problem.

 Game parameters—The resources to be managed are principally the radio resources, alt- hough computing resources (available processing capacity and power) may also be in- cluded. These resources are limited, and may be treated and traded as virtual money in other games. Understanding the dependencies between resources is of educational value and needs to be potentiated.

 Game rules—The game rules will be of the following form: In a single-player game the student acts as the radio operator and has the task to submit a certain amount of data in a defined frequency range. He or she can turn certain knobs that tune the transceiver to generate a specific waveform. The player gets credit for successful transmission and feedback about how well he or she performed in terms of spectral efficiency, throughput, bit error rate (error free transmission is not necessary in communications as a certain amount of bit errors can be tolerated for most if not all communications systems.) The student will have limited time and be allowed to perform a reasonable number of transmission attempts. After the time runs out, post-performance analysis will be provided that may include a video or a live demo about what a better parameter selection would have been for the experienced radio and computing (processing capacity, simulated battery power level, etc.) environment.

In a possible second version of the game, the student will be given the task to provide a (set of) service(s) to one user or a certain number of users. He or she may need to auction spectrum for network deployment through emulated short-term spectrum licensing, for instance. The student will need to make decisions quickly to use the resources in the best possible way. Again the performance is monitored, fed back in real time and analyzed after the game is over. A cognitive engine opponent and or confederate as a Non-Player Character (NPC) provides opportunity and practice to logical reasoning and mediating common biases in the human player. Table I describes how various game phases can load on these biases.

Table I—Game phases. Phase 1: Confirmation Bias Player reasons with respect to gathering disproving NPC-as-Ally evidence or alternative hypotheses Fundamental Player reasons with respect to the motivations of the Attribution Error ally Blind-spot Bias Player reasons with respect to the decision-making of the ally Phase 2: Representativeness Player reasons with respect to probabilities, sampling, NPC-as- and large numbers Adversary Projection Bias Player reasons with respect to the future utility of a current choice Anchoring Bias Player reasons with respect to a “focal” hypothesis

 Tutorials—According to recent studies tutorials should be optional, not mandatory as the true learning experience often comes from trial and error20. This is similar to the fact that we learn something when we are fully exposed to it, rather than by simply following instructions. That is, rather than providing instructions how to play and win the game, we believe that the goals to be achieved should be presented, whereas the knobs and meters should be obvious from the immersive game portal. After a game milestone is achieved— independent of whether successful or not—and after the game is over, the users’ performance is explained, possibly with hints on how to be more successful the next time or, ideally, specific exercises that help understanding what happened in the game and how the improve. Conveying the underlying subject to be taught must not be forgotten in this context.

4.3 Snapshot of tools

Spectrum and signals are invisible. The signal power can be measured in different spectral bands and visualized in different forms. Figure 3 shows two examples.

All relevant radio communications parameters can be made explicitly available to students acting as radio operators. The visual tools and the game elements can help understanding what effect one or a set parameter values have on the spectrum as well as the correct decoding of transmitted information. Figure 4 shows a 3D spectrum screenshot obtained in real time while transmitting a frequency division duplex (FDD) orthogonal frequency division multiplexing (OFDM) signal similar to the one used in 4G.

4.4 Educational sessions

The learning objectives are numerous and range from gaining practical insights into some very basic to advanced communications principles and parameters. These include:  Digital signal processing: modulation, coding, channel estimation and equalization, etc.,  Transmission power,  Bandwidth,  Multiuser channel access, f Each node allows to 2.2 GHz f f • Zoom in or out to get information about 2 GHz spectral occupation at finer or coarser 1.8 GHz granularity • Scroll to visualize 1.7 GHz higher or lower frequencies 1.5 GHz • Dissect, to observe a Radio #2 specific frequency or band  1D heat or 2D Radio #1 Radio #3 contour map.

Fig. 3. Spectrum usage visualization, showing three radios or sensors on one floor with two ways of representing spectrum usage: line graph or color coded.

(a) (b) Fig. 4. FDD-OFDM signal transmitted over the air and captured by a sensing node and represented using different presentations: 3D spectral plot (a), where signal energy is plotted over frequency and time, and overlaid waterfall plot over frequency and time and line plot as a function of frequency (b).

 Interference, and  Quality of service (QoS).

Below we describe three example interactive laboratory sessions that use our tools in the class- rooms.

4.4.1 Session #1: Signals in the spectrum

Synopsis—A short lecture provides a very basic and high-level introduction on the concept of radio resources. As part of this lecture the students access the web-based visualization tools and scan some predefined spectrum bands. The instructor generates signals in some bands. The students comment their observations and discuss them with the instructor until they have all gained an understanding of what the colors (2-D) or hills and valleys (3-D) represent.

Learning Objectives—The objective of this session is very simple: Understand the very basic concept of radio resources (spectrum, transmission power, etc.), spectrum occupation and interference. The final goal of this session is realizing the need for radio resource management.

4.4.2 Session #2: Bandwidth, modulation, and throughput

Synopsis—The students are given the task to observe the spectrum and figure out spectral opportunities. Each group selects a spectral hole or white space to transmit on. The students need to report the available bandwidth and chose a modulation schemes and data rate. The transmission power is first arbitrary and later limited to some threshold.

The students then activate their transmitters and get feedback of the use of spectrum. Students that successfully finish their task help other students resolving the formulas and choosing the right transmission parameters. Several combinations may exist. The students are challenged by, to successfully transmit at highest rate at a given power level and bandwidth and report their results, including bit or block error rate (BER, BLER), throughput and occupied bandwidth and time.

Learning Objectives—The learning goal is to understand the flexibility of SDR transceivers and the various options of adapting the transmission parameters. The second goal is making the students organize the use of spectrum and deploy a transmitter that best adapts to the radio environment. The concept of spectral efficiency is naturally introduced when analyzing the results.

4.4.3 Session #3: Cognitive radio

Synopsis—The students will be players of a game and get instructions how to operate their radios, mimicking a centrally-controlled cognitive radio. The players will observe the spectrum and analyze their transmitter options while waiting for instruction by the network. Once they receive a channel to transmit on for a certain amount of time, they assemble their transmitter as quick as possible and transmit until the timer runs out. Another time they are informed to transmit, a note pops up that they have to vacate the channel immediately. The system monitors their actions to later report how well they performed. The game is over when the data packets are transmitted requiring several transmission time slots.

The instructor then explains, what they just did is centrally-controlled secondary spectrum access and that they were allocated short term “licenses” for the first two transmissions. In the third case a primary user requests regaining the channel. When this happens, the secondary user needs to immediately vacate the channel. Signaling may happen in practice through a cognitive pilot channel.

Learning Objectives—The students’ experiments are designed to help them understand the concepts of primary and secondary users (PUs/SUs), the concept of signaling or control channel, the pros and cons of a centrally controlled cognitive radio networks, and the difference between short-term licensing and opportunistic access. 4.5 Curriculum integration

The curriculum development needs to be cautiously planned for a successful integration of the newly developed teaching methods and tools. The integration is ongoing at two institutions of higher education for the following (junior and senior level) courses:  Introduction to Communications Systems (Virginia Tech)  Software-Defined and Cognitive Radio Design (Virginia Tech)  Digital Communications (Virginia Tech)  Communications II (Weber State University)

The students will use a computer to access the web-based visualization tools and games control- ling the radios of CORNET. The tested nodes are accordingly scheduled for in and out-of class use.

5 Evaluation

Evaluation has two different components: evaluation of tools and evaluation of students. The evaluation and grading of student work may help evaluating the tools. To increase the learning effect as well as get valuable feedback about the teaching method and tools, students may provide a self-evaluation, indicating where they felt they did well and where not during an exercise or game and reason about what they would do differently next time. The instructor can use this to understand the students’ background and level of understanding, before providing more person- alized feedback.

Donovan et al. 21 evaluates the benefits of the gamified course by evaluating benefits—grades, attendance—versus cost of deployment and maintenance. This is a valid metric to merge both evaluation subjects.

5.1 Student evaluation

We believe that students should be assessed based on the reasoning behind their “game” choices and the level of the established learning goals achieved, rather than the actual game score. Stu- dents can therefore be asked to explain their choices or outcomes and present alternative strate- gies. Hence, it is important to integrate this with the lecture and learning goals.

Using empty points and badges to manipulate the students’ decisions to make the right decisions may not be the best option. Rather, students who are able to accomplish meaningful goals, goals that they define and redefine along the learning journey, may have a better effect.

On the other hand, metrics that allow a pseudo-objective or, at least, consistent and fair evaluation of the students’ performances need to be defined. Since the proposed tools are meant to complement the lectures and labs and help achieving the learning goals, another option is to evaluate the students using traditional evaluation techniques—based on homework, exams, presentations, etc., after being exposed to the tools. 5.2 Tool evaluation

The outcome of traditional examination techniques with students who were exposed to the new tools and those who were not will provide some insights about the usefulness of the tools. In- structors may choose or choose not to use these tools in parallel classes or decide when to introduce them. Some insights about the usefulness of the tools can be gained from this.

If traditional student evaluation methods are applied, students may be examined before and after being exposed to the tools. If all other external factors could be eliminated, this would indicate how well the tools prepare the students for their exams, in the short term, and for their profes- sion, in the long term. External factors cannot be fully eliminated, though, so this technique cannot give absolute results.

The students that are exposed to the tools will have the freedom to select their representation of choice as well as switch between representations as many times as desired. By offering different views and measuring which the students preferred or where they spent most of their time for solving one or another task will help evaluating the usefulness of tools and guide future development.

6 Risks

We apply visualization and gamification techniques to wireless communications and introduce them to undergraduate engineering education not to substitute traditional teaching methods, but to complement them. Our hope is that the tools will engage students in the learning process and help grasping complex concepts. However, as pointed out in Section 3 of19, students often have difficulty understanding what the visualization represents. Spectrum is neither visible nor tangible and we provide a computer-aided visual representation of it, based on real measurements and signal processing, called spectrum sensing. In other words, colors, shapes or other visual effects represent physical phenomena, but are not phenomena by themselves. To mitigate this risk, it might be useful to explain how the visual representation are obtained or even let the students develop parts of it. The waterfall plots representing interference temperature, for instance, are simply color-coded versions of measured and processed data samples based on spectrum analysis or sensing techniques.

Gamification in education has the additional risk of losing the link to educational, moving to the background. Developing highly-engaging games is the objective of every game developer, but for a game to be engaging abstractions and inventions may be needed. Such games can become an addiction, whereas an educational game would most probably not. Once the taught concepts are fully mastered by the student, the purpose of the educational game has been achieved and there is no point in continuing playing the game. In other words, there is a tradeoff between developing sophisticated visualization and gaming options, which may mask or even modify physical meanings, and simplistic tools that do not present sufficient challenges or options for the student to engage. Independent of the level of sophistication, what physical resource is represented, needs to be properly conveyed.

Gaming can easily distract from the educational goal, either because it is too exciting/abstract or too easy. Similarly, if the tools become too sophisticated, the physical meaning of actions and reactions might become too abstract. This means that the games need to be carefully crafted and should open a new window for knowledge acquisition in wireless communications.

The game outcome is not the score, but rather the educational effect it has on the students who are the players of the game. The player will receive statistics indicating well he or she performed and how the performance can be improved. Understanding why the game was won or lost and what can be improved is what the student should take from the game. The game may accomplish capturing the players’ attention for playing it again and again until the goals are achieved and understood. The player may then pass to the next level of the game or get new assignments that require consulting text books or other resources before being able to continue with the game.

7 Conclusions and outlook

This paper presents our contribution to electrical engineering education. We suggest enhancing undergraduate communications courses to make them more interesting, tractable, and effective. Our approach is combining visualization with gamification techniques to illustrate abstract concepts to students. The main concept is providing students with a flexible wireless communications testbed and tools that facilitate experimenting and experiencing cause and effect by using different forms of visualization. Gamification provides extra motivation by defining tasks to be resolved by means exploration. We are currently integrating the tools in different undergraduate communications courses at two schools while continuously developing and refining them. These tools will be available for use from9 and will be portable to other testbeds. Our long-term goal is engaging more students and helping them understand critical communications principles and emerging wireless technologies to be able to use them effectively in R&D or other professional contexts.

Acknowledgment

This work is supported by NSF grant award #1432416. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References

1. S. Deterding, “Gamification: designing for motivation,” ACM Interactions Magazine, Vol. 19, Iss. 4, July- Aug. 2012, pp. 14-17. 2. A. Iosup, D. Epema, “An experience report on using gamification in technical higher education,” Proc. SIGCSE’14, Atlanta, GA, USA, 3-8 March 2014. 3. M. McHenry and K. Steadman, “Spectrum occupancy measurements location 1 of 6: Riverbend Park, Great Falls, Virginia,” Share Spectrum Company, Tech. Rep., 2005. 4. P. Sutton, L. Doyle, “The gamification of dynamic spectrum access & cognitive radio,” Proc. IEEE Global Engineering Education Conf. (EDUCON), Istanbul, Turkey, 3-5 April 2014, pp. 107-113. 5. E. Baccelli, F. Juaschek, O. Hahm, T.C. Schmidt, H. Will, M. Wahlisch, “The MANIAC challenge at IETF 87,” IETF Journal, Vol. 9, Iss. 2, Nov. 2013. 6. A.C. Marcum et al., “Low SINR synchronization for the DARPA spectrum challenge scenario,” 2014 IEEE Military Communications Conference (MILCOM), Baltimore, MD, pp. 1447-1453. 7. M. J. Callaghan, K. McCusker, J. L. Losada, J. Harkin, S. Wilson, “Using game-based learning in virtual worlds to teach electronic and electrical engineering,” IEEE Trans. Industrial Informatics, Vol. 9, No. 1, pp. 575-584, Feb. 2013. 8. P. Sutton, L. Doyle, “Spectrum wars - gamification of dynamic spectrum access and cognitive radio,” Proc. IEEE Dyspan 2014. 9. Cognitive Radio Network Testbed (CORNET) Web Site, http://www.cornet.wireless.vt.edu/ 10. The Networking and Information Technology Research and Development (NITRD) Program - Wireless Spec- trum Research and Development (WSRD) Senior Steering Group Web Site, http://www.nitrd.gov/apps/wsrdtestbedinventory/index.php?title=WSRD_Testbeds 11. Ettus Research Web Site, www.ettus.com 12. J. Behr et al., Dynamic and interactive aspects of X3DOM, Proc. 16th International Conference on 3D Web Technology. 2011, ACM: Paris, France. p. 81-87. 13. Behr, J., et al., A scalable architecture for the HTML5/X3D integration model X3DOM, Proc. 15th Interna- tional Conference on Web 3D Technology. 2010, ACM: Los Angeles, California. p. 185-194. 14. Marion, C. and J. Jomier, Real-time collaborative scientific WebGL visualization with WebSocket, Proc. 17th International Conference on 3D Web Technology. 2012, ACM: Los Angeles, California. p. 47-50. 15. T. Franke, et al., “VCoRE: a web resource oriented architecture for efficient data exchange,” Proc. 18th Inter- national Conference on 3D Web Technology. 2013, ACM: San Sebastian, Spain. p. 71-78. 16. K. Moreland, “Diverging Color Maps for Scientific Visualization Advances in Visual Computing,” G. Bebis, et al., Editors. 2009, Springer Berlin / Heidelberg. p. 92-103. 17. D. Borland, R.M. Taylor, “Rainbow Color Map (Still) Considered Harmful,” IEEE Computer Graphics and Applications, Vol. 27, Iss. 2, 2007. 18. D.A. Norman, Cognitive Engineering, in User Centered System Design, D.A. Norman and S.D. Draper, Edi- tors. 1986, Lawrence Erlbaum Associates: Hillsdale, NJ. p. 31-61. 19. John K. Gilbert, Miriam Reiner, Mary Nakhleh (Eds.), Visualization: theory and practice in science education. Models and Modeling in Science Education, Vol. 3, Springer Science+Business Media B.V., 2008. 20. Jane Bozarth, “Alicia Sanchez: A review of recent games research,” ACM eLearn Magazine, Sept. 2013, http://elearnmag.acm.org/archive.cfm?aid=2532328. 21. S. O’Donovan, J. Gain, P. Marais, “A case study in the gamification of a university-level games development course,” Proc. SAICSIT '13, East London, South Africa, 7-9 Oct. 2013.