THE EVOLUTION and IMPACT of a METEOROLOGICAL OUTREACH PROGRAM for PUBLIC SAFETY OFFICIALS an Update on the Oklahoma Mesonet’S OK-First Program

THE EVOLUTION AND IMPACT OF A METEOROLOGICAL OUTREACH PROGRAM FOR PUBLIC SAFETY OFFICIALS An Update on the Oklahoma Mesonet’s OK-First Program

James E. Hocker, Andrea D. Melvin, Kevin A. Kloesel, Christopher A. Fiebrich, Robert W. Hill, Richard D. Smith, and Steven F. Piltz

Weather education and data delivery to the public safety community have been the hallmark of the OK-First program for the past 20 years

klahoma is no stranger to life-threatening 2017). This history of disasters proved instrumental in weather conditions. A review of Federal Emer- the conceptualization of an automated meteorological Ogency Management Agency (FEMA) presiden- surface observation network capable of monitoring tial disaster declarations (1953–2017) reveals that Oklahoma’s rapidly changing weather conditions. Oklahoma is ranked first in disaster declarations The concept was that through the development of a categorized as “severe storms” and currently has the network providing near-real-time surface data across third-most disaster declarations of any state (FEMA the state, a variety of users would be armed with high-quality weather information to support critical

AFFILIATIONS: Hocker, Melvin, Kloesel, and Fiebrich—Okla- decision-making during hazardous weather events. homa Mesonet, Oklahoma Climatological Survey, University of Commissioned on 1 January 1994, the Oklahoma Oklahoma, Norman, Oklahoma; Hill—Stillwater Office of Emer- Mesonet (Brock et al. 1995; McPherson et al. 2007) gency Management, Stillwater, Oklahoma; Smith—NOAA/NWS/ is the state’s automated surface observation network. Weather Forecast Office, Norman, Oklahoma; Piltz—NOAA/ The Oklahoma Mesonet (referred to as Mesonet) NWS/Weather Forecast Office, Tulsa, Oklahoma monitors a variety of surface weather conditions CORRESPONDING AUTHOR: James Hocker, [email protected] including air temperature, relative humidity, air The abstract for this article can be found in this issue, following the pressure, wind speed, wind direction, rainfall, and table of contents. solar radiation at 5-min intervals as well as subsur- DOI:10.1175/BAMS-D-17-0100.1 face conditions including soil temperature at 15-min In final form 2 March 2018 intervals and soil moisture at 30-min intervals. With ©2018 American Meteorological Society For information regarding reuse of this content and general copyright a total of 120 stations, Mesonet sites are located in information, consult the AMS Copyright Policy. all of Oklahoma’s 77 counties at an average station spacing of 29 km.

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2009 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Mesonet data have been shown to benefit a wide data with an information system that would benefit range of sectors spanning agriculture, drought moni- these critical public safety users—the OK-First pro- toring, climate monitoring, public safety, wildland gram was the answer to that need. fire management, short-term forecasting, state and OK-First began in 1996 as a U.S. Department of federal agencies, residential and public users, energy Commerce telecommunications infrastructure grant. users, K–12 education, and research applications The concept of OK-First was simple but empower- (Ziolkowska et al. 2017). In the case of several of these ing—to provide eligible public safety officials with 1) a sectors, dedicated outreach programs were developed means for receiving information (e.g., a computer and by Mesonet staff with the intent of providing relevant Internet connection), 2) free password-protected ac- Mesonet data products and tools, education, and cess to meteorological data, and 3) basic meteorologi- expertise to address needs of different user groups, cal training to help in the interpretation of a variety including K–12 education (McPherson and Crawford of weather and radar products. With Mesonet, radar, 1996), public safety (Morris et al. 2001), agriculture and NWS data as the backbone of the system, OK- (Klockow et al. 2010), and wildland fire management First was created to provide public safety officials with (Carlson and Bidwell 2008). While the data and tools information that was previously unavailable to them provided through the Mesonet’s four dedicated out- for a variety of reasons (e.g., lack of computer, pro- reach programs have produced a variety of benefits hibitive cost of radar data). The training component to different users over the years, nowhere else has of the program was unique since weather and radar it proven to be so crucial to life-or-death decision- data interpretation classes for nonmeteorologists were making as it is has been to its public safety outreach essentially nonexistent at the time, with the excep- program [referred to as Oklahoma’s First-Response tion of NWS storm spotter training (Doswell et al. Information Resource System using Telecommunica- 1999). Following the development of a data website tions (OK-First)]. and training content, participant recruitment, and The purpose of this paper is to provide an over- a competitive application process, OK-First classes view of the Mesonet’s OK-First program. With the began in the summer of 1997. program recently reaching its twentieth anniversary, this paper serves as an update to past publications on PARTICIPANT TRAINING. Scientific infor- OK-First (Morris et al. 2001, 2002) and will highlight mation by its very nature is inherently complex yet the evolution of the program. Specific focus will be powerful—this is particularly true for meteorological given to program history, participant training, data information. For this reason, staff at the Mesonet access technology, and program impact. determined from the onset of OK-First that comprehensive training would be a fundamental element of HISTORY. With the commissioning of the Mesonet the program to provide program participants with in 1994, an abundance of meteorological informa- a thorough introductory education in meteorology, tion became available in Oklahoma. The data had data interpretation, and technology for viewing data. immense value in monitoring hazardous weather This section will cover a variety of topics relating to events; however, the mere presence of Mesonet data OK-First training and its evolution over the last two alone did not address recurrent deficiencies docu- decades, including program participation, classes, mented in several National Oceanic and Atmospheric the importance of feedback, training changes, and Administration (NOAA) service assessments from class attendance. flooding (NOAA 1977, 1991), heat (Changnon et al. 1996), and tornado events (NOAA 1994). The assess- Program participation. Prospective OK-First partici- ments found that a lack of weather data dissemination pants must meet several eligibility requirements (e.g., systems rendered the issuance of National Weather be in a public safety–related position) and agree to Service (NWS) forecast and warning information program rules specified in the program participant useless since public safety officials were unable to agreement (www.mesonet.org/index.php/okfirst receive the information in a timely manner. Based /participant_agreement) before being permitted to on these previous events, scientists at the Oklahoma enroll in classes. While emergency managers, fire Mesonet recognized that without modern computing officials, and law enforcement were the initial target technology and a dedicated information dissemina- audience of the program, interest has spread to other tion system, public safety officials would not be able to weather-impacted decision-makers. Over the last 10 receive the wealth of data now being collected by the years, the list of participating groups has grown and Mesonet. There was a distinct need to pair Mesonet diversified and now includes tribal officials, city and

2010 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC county leadership, health officials, school officials, recertification classes began in 1998, were two days military officials, voluntary organizations active in in length, and consisted of a review of many of the disasters, and state and federal agencies. meteorological concepts covered in the certification class as well as new laboratory exercises. The premise Classes. Training has been the bedrock of the OK- behind the continual training approach was similar to First program since the program’s inception and is that of recurring weapons training for officers—rou- a required element to gain password-protected ac- tine time “at the range” is needed to maintain skills. cess to the tools provided by the program. Training This learn-through-repetition approach has proven is initially provided through an intensive multiday to be very successful, if not crucial, as it has provided certification class, which over the years has evolved an avenue for updating the community of users as in content and duration. During the 1997/98 period software/tools change, new datasets become available, of the program, the certification class had a threefold and new topics are introduced in classes. Because of purpose: 1) to introduce officials to modern com- user feedback, recertification classes were shortened puting technology, 2) to train them on important to one day beginning in the fall of 2004. meteorological concepts, and 3) to close the data gap While the certification and recertification classes between the NWS and public safety community. The formed the initial training offerings of OK-First, first portion of the class was devoted to technology substantial class feedback during the first several training (e.g., how to use a computer, mouse, the years of the program identified a distinct need for Internet) and for a number of participants was their an additional class type. Many of the initial partici- first exposure to computers and using the Internet. pants to OK-First certification were city or county This computer training was limited to the first emergency management directors who relied on three classes (1997/98) and largely served to train additional staff and volunteers, particularly during attendees who were selected to receive a brand new active weather situations. A common request was the computer as part of the training. The final five days need for a radar interpretation class specifically for of the class were devoted to meteorological train- assistants and volunteers that supported emergency ing and were accomplished via a combination of management operations. Since many assistants are presentations on a variety of topics (e.g., Mesonet either unpaid or only part time, taking five days data, radar data, basic meteorology, terminology) off to attend the certification class was not an op- and computer-based laboratory exercises that em- tion. To meet the expressed demand, Mesonet staff phasized data and product interpretation using developed the OK-First assistant certification class OK-First tools. Laboratory exercises focused on a and held the first one in spring 2000. The 2-day class variety of weather event types (e.g., severe storms, focused exclusively on radar and the severe storm winter weather, fire) and could include anything portions of the certification class. To be eligible for from locating a dryline on a surface map to identi- the assistant certification class, participants were fying possible winter precipitation types based on required to be approved by (and assistants to) an radar reflectivity appearance and Mesonet data. active, certified OK-First participant. Assistant Instructors for the classes included meteorologists certified participants initially were not required from the Mesonet as well as the Norman and Tulsa, to take any additional OK-First training; however, Oklahoma (OK), NWS offices. The involvement of following participant feedback and a program rule NWS staff in the classes (Figs. 1b,f) was particularly change in 2015, these participants are now required crucial as it further legitimized OK-First training, to attend recertification training no less than once facilitated relationship building between public every 24 months. safety officials and the NWS, and allowed NWS staff to see what meteorological data public safety The importance of feedback. The single-most critical officials were now able to access through OK-First. element responsible for the success and longevity Because the concepts covered in OK-First certifica- of OK-First has been the user-driven design of the tion were complex while the technology and science program. Mesonet staff understood that to best serve continued to advance, Mesonet staff implemented the community meant to continually listen to user a training model that avoided a “one and done” feedback and implement suggestions as feasible. This approach. The concept was simple: certified OK- “feedback first” approach has influenced all aspects First participants were required to attend routine of the program, especially its training. This has been recertification classes at least once every 18 months done informally in classroom settings through class to maintain certification and data access. OK-First discussions as well as more formally via postclass

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2011 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Fig. 1. Photographs from OK-First classes at a variety of locations over the years, including (a) Sarkeys Energy Center at the University of Oklahoma (OU) in 1999, (b) the Warning Decision Training Branch in 2006, (c) the Moore Public Safety Center in 2014, (d),(e) the Radar Innovations Laboratory at OU in 2016, and (f) the National Weather Center at OU in 2017.

and periodic postseason evaluations. The variety of representing a variety of different roles (city, county, approaches has yielded invaluable feedback on topics state, tribal, K–12, university, regional, and federal) users wanted to learn more about, elements of classes as well as relevant partners (e.g., NWS and FEMA), that were valuable or needed adjustment, preferred the committee gathers annually for a 1-day meeting instructional methods, and technical suggestions on to provide feedback on a variety of topics spanning OK-First websites and software. training, recruitment, technology, program policies, To augment feedback provided at classes and and other topics. With the exception of the committee provide an opportunity for more in-depth dis- seats for non-OK-First participants (e.g., NWS and cussions, Mesonet staff established a 15-member FEMA), committee membership is limited to three OK-First Advisory Committee in 2011. Composed years, which allows for new voices and perspectives of active OK-First participants (no fewer than 10) on the committee over time.

2012 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Table 1. Instructional topics and activities for each OK-First class as of 2017. Assistant Recertification Certification certification class class class (1 day or Instructional topic/activity (4 days) (2 days) online) Introduction to meteorology P P Measuring the atmosphere P P Radar fundamentals and interpretation P P NWS severe weather operations P NWS Storm Prediction Center products P P Rotation of Severe storms P P topics, new products and Lightning P technology, up- Flooding P dates to website Fire weather P and software, and new labora- Winter weather P tory exercises Website and software training P P using software Laboratory exercises P P Panel discussion with NWS and media P Warning scenario with NWS P Tour of facilities P P

Training changes. With the collection of feedback mech- OK-First participants have had access to a variety anisms in place (i.e., the OK-First Advisory Committee, of meteorological reference materials on a password- in-class discussions, and postclass evaluations), Me- protected web page since the late 1990s; however, sonet staff receive a steady stream of ideas to improve users began expressing a greater interest in online the program. Over the past five years, suggestions have classes in the 2010–11 time frame. To address the been made on a variety of training elements includ- increased demand for online recertification training, ing increasing the educational standard and rigor of Mesonet staff integrated an open-source learning the program, expanding to new methods of training management system (Dokeos; www.dokeos.com) delivery, classroom improvements, and class content. into a newly redesigned OK-First web page in early Program rigor was increased through the imple- 2014. The first online OK-First recertification class mentation of a testing component at the certification was offered in spring 2014 and was conducted in a and assistant certification classes beginning in spring weeklong, asynchronous format. The format and 2014. All new participants to OK-First training are design were well received and remain the same today, required to score a 70% or higher on a comprehensive with 392 students successfully completing 19 online end-of-class test to receive their certification. Each recertification classes through 2017. Traditional in- student is allowed up to one retake as needed before person recertification classes are still offered and well being required to take the class again. Also relating attended, but for those participants who have difficult to the increased educational standard, required web- work schedules or prefer a self-paced class, the online based prerequisites were added to both the certifica- recertification classes have been a valuable addition. tion and assistant certification classes in spring 2016. Additional requests for offering the certification and The 2.5 h of prerequisites include a class pretest as well assistant certification classes in an online format have as introductory presentations on meteorology and been made as well; however, to date, these classes meteorological measurements, which previously were remain exclusively in the in-person format because covered in class. Establishing these prerequisites has of the importance of relationship building between allowed for more interactive sessions during class and public safety officials and the Mesonet, NWS, and has also resulted in fewer nonattendees to training as media. enrollees are more vested given they have completed In terms of classroom improvements, Mesonet a portion of the content beforehand. staff have implemented several changes over the last

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2013 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC addition to helping students anonymously assess their understanding of different topics, the system provides immediate feedback to instructors on the effective- ness of their teaching. In fall 2016, in an effort to modernize the delivery of class materials at in-person classes, Mesonet staff began providing class materials exclusively in a digital format (e.g., single PDF file with simple bookmarked navigation) with distribution occurring in advance of class. This replaced a long- standing method of providing class materials in both print (hundreds of pages) and a series of electronic presentation files, which were provided upon arrival at the class. Student feedback on the new digital-only format was overwhelmingly positive and has helped to reduce program costs. Though the core focus of OK-First training has re- mained radar and Mesonet data-centric throughout the 20-yr history of the program, class materials and content have slowly evolved over the years. Training materials are routinely modified to update graph- ics; include new technolo- gies (e.g., dual-polarization radar), new products (e.g., Fig. 2. The 1997–2017 OK-First annual (a) training totals of all classes Mesonet wet-bulb globe combined (certification, assistant certification, recertification, and online temperature), and new tools recertification) and online training (recertification only) and (b) training (e.g., updated website); and totals by class type. incorporate new laboratory exercises. Current meteoro- few years to modernize the educational experience logical topics covered in OK-First classes (Table 1) of its classes. Beginning in fall 2013, Mesonet staff include all relevant hazards for Oklahoma and also began experimenting with student response devices comprise topics that users have specifically requested as a tool for reviewing concepts during presenta- (e.g., lightning). tions. The devices were an immediate hit and were In spring 2015, an experimental NWS and media quickly integrated into all class presentations. In panel discussion was added to the certification class

2014 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC to facilitate a conversation between the public safety In terms of changes in training numbers over officials and their counterparts in the integrated time, annual class totals have increased steadily, as warning team (Doswell et al. 1999). Composed of evident in Fig. 2. four broadcast meteorologists from different media Factors influencing the increase in annual train- markets and two NWS meteorologists from different ing numbers include 1) the addition of the assistant offices (Fig. 1e), the question-and-answer-style panel certification class in 2000, 2) increased program vis- discussion provided a forum for discussing a variety ibility over time through the Mesonet’s attendance of topics that the public safety officials initiated. Some at the annual Oklahoma Emergency Management of the topics discussed included message consistency, Association (OEMA) Conference and the annual media “event hyping,” and methods for improving Inter-Tribal Emergency Management Coalition communications. The session was so popular that Summit, 3) inception of the OK-First Advisory Com- it immediately became a permanent addition to the mittee in 2011, 4) addition of online recertification certification class curriculum. classes in 2014, 5) a program rule change requiring assistant certified participants to attend recertifi- Class attendance. Over the last two decades, Mesonet cation classes no less than once every 24 months staff have trained more than 1,400 public safety of- beginning in 2015, and 6) members referring others ficials at a total of 42 certification and 58 assistant to the program. certification classes. Following initial certification, OK-First participants have subsequently attended a DATA ACCESS TECHNOLOGY. While train- total of 186 recertification classes with a total class at- ing formed the foundation upon which OK-First was tendance of 2,951. In total, OK-First classes from 1997 built, technology offered by OK-First has provided to 2017 amassed a total attendance of 4,579 people the incentive for public safety officials to join and taught at 49 different locations across Oklahoma. remain in the program. During the early years of

Fig. 3. Original OK-First current data web page from 1997 displaying interactive maps from (a) the radar section and (b) the Mesonet section.

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2015 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC OK-First weather data web page. The initial OK-First weather data web page (launched in 1997) was a cutting-edge weather data portal that provided users with access to radar data, Mesonet data, NWS products, and other products. A subset of products on the page—namely, radar and Mesonet data—were available as powerful interactive maps. Through the incorporation of a special Mesonet-developed web browser plug-in called WxScope, these interactive maps (1997; Fig. 3) could be zoomed, panned, and animated, thus providing users with an ability to tai- lor the maps to their needs. The dissemination technology was innovative in that raw radar and Mesonet text data files were transmitted to the user and subsequently visualized on the client’s machine. This allowed significant amounts of data to be transferred via the

Fig. 4. Early 2000s OK-First enhanced briefing web page featuring (red banner web without the overhead across the top) customizable alerts, (top left) interactive radar reflectivity mo- of large postprocessed files saic with NWS watches and warnings, (top middle) data from a user-defined (e.g., image and animation Mesonet site, (middle left) interactive regional wind chill map, (bottom left) files). NWS Storm Prediction Center Day 1 Convective Outlook, and (bottom right) Participant feedback NWS Zone Forecast for a user-defined county. was solicited at every OK- First class and led to sev- the program, OK-First provided many public safety eral major improvements to the page. One of the officials with their first introduction to the informa- first updates to the page was the introduction of tion age. While new meteorological datasets were user-customized preferences in 1998, which allowed becoming available during the mid-1990s, meager participants to customize their interactive maps to budgets and high subscription costs for some datasets display or hide different geographic layers in their (e.g., radar data) made it impossible for most public plug-in-enabled maps (e.g., county borders, different safety officials to access meteorological information. road types, rivers, Mesonet site locations). All of these OK-First was the answer to the cost-prohibitive nature settings were saved and applied for future use of the of accessing meteorological data, which the program page. Users also expressed an interest in having an achieved through providing select participants with a option between a one- and two-screen mode to dis- free computer (1997–98 only) and a free weather web play data on the website. This capability was added in page for viewing a variety of datasets (1997–current). 1998 and provided users with control over what would In this section, a summary of OK-First’s data access be displayed in the two screens (e.g., radar, Mesonet, technology will be provided. NWS, or other data). Another user-inspired update

2016 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Fig. 5. The 2007 redesign of the OK-First current data web page. Displayed products include (top middle) an interactive radar reflectivity mosaic with NWS watches and warnings, (top right) an interactive dewpoint temperature map, (bottom middle) an NWS Storm Prediction Center Day 1 Convective Outlook, and (bottom right) an NWS Zone Forecast for a user-defined county.

occurred in 1999 and featured a new way of organiz- interactive map products, 3) an ability to view data ing data on the page into six different threat types— full screen, 4) an alerting function for up to seven thunderstorms and tornadoes, flooding, tropical user-specified counties, and 5) a capability for user- cyclones, ice and snow, heat and drought, and fires built WeatherScope maps that could be uploaded to (e.g., Fig. 4 in Morris et al. 2001)—as well as dividing the page. In addition, the page had a new four-screen the data into subsections based on their utility before, mode that served as the initial data view when logging during, and after events. This greatly assisted users in to the page (Fig. 5). in navigating to the most appropriate maps given the In the years following the 2007 update, rapid weather situation and began a transition from a pure growth in mobile web technology created challenges information-support system to a decision-support for the future sustainability of the OK-First weather system. Around this time, interpretation aids were data page. Not only was the page not designed for also integrated alongside the threat and data products mobile platforms, but web browser support for plug-in to provide educational opportunities for users view- software was decreasing. This posed a major threat ing the data. Subsequent minor updates to the page to the OK-First website given the page relied heavily were made in the early 2000s including an enhanced upon the WeatherScope plug-in for all interactive briefing mode that allowed users to customize the mapping. In 2012, the Mesonet team began develop- zone forecast location, display data from a specific ment of a new OK-First weather data page—a design Mesonet site, and receive alerts for a location of their that for the first time in the history of the page would choice (Fig. 4). not rely on Mesonet-developed plug-in software for A significant update to the OK-First weather data interactive mapping. Instead, the new design lever- web page was completed and released to users in 2007. aged JavaScript and the growing visualization capa- The change brought about a major redesign to the bilities of web browsers. page, which featured 1) a hazards-based menu system, Following several years of development, internal 2) full integration with the Mesonet’s WeatherScope evaluation, and external testing with OK-First par- software (McPherson et al. 2007) as a plug-in for all ticipants, the new OK-First Weather Briefing web page

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2017 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Table 2. New features and capabilities of the OK- In addition to these familiar features, a number of new First Weather Briefing web page (launched in 2016). features and capabilities were unveiled, as specified in A completely customizable one-, two-, three-, or four- Table 2. The customizable data display allows users to panel display drag and drop products from the menus at the top (e.g., A capability to save collections of products (briefings) for under the WX Hazard, WX Data, and My Products future use sections) into different windows on the page, thus Customized alerting for up to 15 counties providing each OK-First participant with a unique dis- Improved data organization that removed redundant play that can be tailored to meet their changing needs product listings depending on the weather situation. Figure 6 displays A search function for finding products the OK-First Weather Briefing web page configured to monitor fire weather conditions during the major A single-panel mobile version for smaller-screen devices Anderson Creek fire event on 23 March 2016. A new push method of data delivery for interactive maps and advisories using the Extensible Messaging and Presence RadarFirst. Protocol (XMPP) Following the 2007 update to the OK- First weather data web page, program participants made requests for a stand-alone software applica- was launched in early 2016. The page offers many of tion to view radar data exclusively. While interac- the popular features of its predecessors including in- tive radar data were available in the updated web teractive radar and Mesonet products, customizable page, users were interested in a companion tool user preferences, multiple windows to view data, and that was faster and easy to use and did not require a sizable collection of more than 300 products to view. a web browser. Software developers at the Mesonet

Fig. 6. The 2016 redesign of the OK-First weather briefing web page viewed during the Anderson Creek fire event on 23 Mar 2016. Products displayed include (top left) an interactive relative humidity, wind, and Dodge City, Kan- sas, base reflectivity tilt 1 product; (top right) Mesonet Burning Index map; (bottom left) visible satellite image; and (bottom right) fire danger widget with data from the May Ranch Mesonet site in northwestern Oklahoma.

2018 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC developed a solution that would leverage off of the mode, dual-polarization radar data, archived data radar portion of the WeatherScope software code mode (used extensively for training scenarios), up- base—a new radar-only software package entitled dated authentication system to handle new individual RadarFirst. accounts, touch-screen compliance, improved map RadarFirst was immediately embraced by the navigation controls, multiple instance capability OK-First community following its launch in 2009 (e.g., the software can be opened multiple times on and to date is one of the most popular tools that OK- the same device), keyboard shortcuts, screenshot First has provided the public safety community. The feature, and lightning data. Several similar radar initial version provided interactive radar data in a software options exist today; however, RadarFirst single window that could be zoomed, panned, and continues to be very popular with the OK-First com- animated. Users could switch between radar products munity as it provides participants with a no-cost and (e.g., reflectivity, velocity, radar-estimated rainfall, no-subscription solution for viewing real-time radar, vertically integrated liquid) and also had an ability advisory, and lightning data as well as past data from to change radar sites. Though this initial version was a sizable catalogue of saved historical weather events. somewhat limited in its features, the simplicity, speed, An example of the software in use during the 16 May and reliability of the software were key in its rapid 2017 severe weather event in western Oklahoma is adoption by the OK-First community. shown in Fig. 7. While RadarFirst was well received initially, participants had and continue to have a wealth of ideas PROGRAM IMPACT. A formal evaluation of OK- for improving the software with new features that First was completed after the first three years of the further support their myriad of public safety respon- program and is summarized in Morris et al. (2001). sibilities. Between 2009 and 2017, a total of 10 versions More recently, an optional short survey (University of of the software have been developed, tested, and Oklahoma Institutional Review Board 8061) was dis- released for Windows (9 versions for Mac). Features tributed to all current OK-First members during May added through these updates include detailed road 2017 to assess program improvement and impact. data, warning polygons and text, spotter network The survey covered several topics including years data, audible alerts for user-set counties, two-screen of experience, most valuable aspect of the program,

Fig. 7. The OK-First RadarFirst software displaying (left) base reflectivity at 0.5° and (right) base velocity at 0.5° from the Frederick, Oklahoma (KFDR), radar at 2359 UTC 16 May 2017. Also shown are severe thunderstorm warning polygons (orange), tornado warning polygons (red), storm-track forecasts (white lines with icons), lightning strike data (white plus and minus icons), and tornado warning text (pop-up window).

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2019 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC most important changes to training, most important Survey participants were asked a similar question changes to technology, and examples of how data and regarding the most important technology improve- training were used. In this section, results of the 2017 ment during the history of the program: 47.3% chose survey as well as statements from several OK-First the development of RadarFirst, 27.4% chose the weather partners will be provided. provision of weather-related mobile apps (e.g., the Mesonet and RadarScope apps), 14.4% chose devel- 2017 OK-First participant survey. A total of 151 re- opment of the new mobile-capable Weather Briefing sponses (146 complete and 5 partially completed page, 6.8% chose the switch from shared agency ac- surveys) were recorded out of 618 active members count logins to individual logins, and the remaining who received the survey, which represents a 24.4% 4.1% chose other (commonly “all of the above”). The response rate. Survey respondents ranged in OK-First selection of RadarFirst (which was user requested in experience from new members to 20-yr participants, 2007) as the top technology improvement reinforces with the average survey respondent having 7.66 years the importance of listening to and being responsive of OK-First experience; 115 respondents (77.7% of to user needs. The selection of weather-related apps total) indicated they had received OK-First certifica- (including RadarScope, which has been purchased tion training, while 32 respondents (21.6% of total) and provided to program participants at no cost to received OK-First assistant certification training. Job them since 2014) as the second-highest technology types of the respondents are shown in Table 3. improvement is also noteworthy. Many of the under- When asked what the most valuable component lying data dissemination and visualization technolo- of OK-First was, 39% chose “training,” 37.7% chose gies developed by Mesonet staff for OK-First in the “technology,” and 23.3% chose “connections” (with late 1990s to early 2000s later laid the foundation NWS and other public safety agencies). Focusing fur- for the widely popular RadarScope application for ther on training, participants were asked to identify mobile and Mac devices (M. Wolfinbarger 2017, the most important training improvement during the personal communication); thus, the technological history of the program: 48.6% chose changing program impact of the program extends well beyond the OK- rules to require all participants (certified and assistant First community. certified) to attend recertification classes, 34.3% chose In the final survey question, participants were the addition of online recertification classes, 9.6% given the opportunity to share a weather event chose requiring all new participants to pass an exam example or a statement on the program. A total to be certified, 4.1% chose adding interactive clicker of 72 survey participants provided open-ended questions to presentations, and the remaining 3.4% responses. The following list highlights several of indicated other (commonly “all of the above”). the statements:

Table 3. 2017 OK-First participant sur- “The training I received from the OK-First program vey respondents by job type (n = 148). was critical in the way that I managed the state’s medi- Fraction cal response to the [enhanced Fujita 3 (EF3)] tornado Role of total that hit El Reno on May 31, 2013. I was able to use the Emergency management 45.3% RadarFirst platform along with my training to safely Fire 13.5% and effectively guide Emergency Medical Service Law enforcement 8.1% (EMS) resources and responders from various locations across Oklahoma to the areas directly affected by 911 dispatch 8.1% the storms. This meant re-routing EMS units several School 4.0% times to keep them out of the path of severe storms. Tribal 4.0% Additionally, I frequently use OK-First data to keep my City 3.4% partners informed before and during weather events Health/medical 3.4% all across Oklahoma” (Darrell Eberly, emergency Volunteer/storm spotter 3.4% manager, Oklahoma State Department of Health). State 2.0% Nonprofit 2.0% “Due to our location in western Oklahoma we often did not get good coverage from the available Other 1.4% news services prior to OK-First. With OK-First and Military 0.7% now especially RadarFirst we can storm spot more Federal 0.7% efficiently and with less personnel in the field in

2020 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC these cases. I believe we now provide much better strikes we have suffered, but snow, ice, heat, lightning, protection to our city and the other communities rain, drought. The data we receive and the training to downstream from the storms due to the capabilities properly use it is indispensable. Every day!” (Gayland of OK-First. I would hate to ever have to go back to Kitch, emergency manager, Moore, Oklahoma). blindly spotting a storm without the information that is provided by OK-First” (Terry Stone, retired Statements from OK-First weather partners. Separate fire chief, Elk City Fire Department). from the 2017 survey, NWS and media partners that have participated in OK-First classes also provided “On April 26, 2016 and May 9, 2016 severe thunder- their impressions and perspectives on the program. storms developed and moved over portions of the The following are several provided statements: city of Oklahoma City. I used my OK-First training, applications, and data to provide decision making “Thanks to the intensive training provided through support to 911 regarding the activation of the new the OK-First program, emergency managers and Oklahoma City outdoor warning siren sectors. The public safety officials in our area of responsibility knowledge, skills, and abilities gained from OK- are probably some of the most weather-savvy in the First were invaluable to interpreting the data and country. They understand more of the language of analyzing the developing situation. As a result, only meteorology and radar, and that enhances the com- one sector was activated on April 26th and only one munication between the National Weather Service on May 9th significantly reducing over-warning or and these local decision makers” (David Andra, false warnings” (Frank Barnes, emergency manager, meteorologist-in-charge, NWS Weather Forecast Oklahoma City, Oklahoma). Office, Norman, Oklahoma).

“In Grady County, the [emergency management “OK-First training gives emergency management and (EM)] office uses volunteer fire departments, decision makers in Oklahoma a unique opportunity municipal police officers, and sheriff’s deputies to to learn important meteorological and radar topics on provide spotting information for the emergency a level that is not available in most other states. The operations center. I use the OK-First products to understanding of various weather-related topics is in- advise them on what is going on and what to look creased through this hands-on training, which surely for. The OK-First program is an indispensable tool improves the collaboration process between OK-First for the emergency management community. We use attendees and the NWS offices that serve Oklahoma” these tools daily as the Oklahoma weather changes (Ed Calianese, warning coordination meteorologist, almost daily. We have the weather briefing page NWS Weather Forecast Office, Tulsa, Oklahoma). up on two screens in our [emergency operations center (EOC)] and county dispatch 24 hours a day. “Routine interaction with weather-savvy emer- Without the OK-First training and usage our county gency officials through the OK-First program would be in serious jeopardy to warn responders and has helped guide the development of new forecast the residents of impending severe weather” (Dale services and web page content. Feedback from OK- Thompson, emergency manager, Grady County First participants has also helped in gaining a better Emergency Management). understanding of how our severe and fire weather outlooks are used in daily planning and decision “The ice storm of January 2010 was a major winter making within the emergency management com- event [that] impacted our city with major damage to munity” (Bill Bunting, chief of forecast operations, electrical distribution infrastructure, which took 12 NWS Storm Prediction Center). days to make repairs before any power was restored. Due to OK-First and NWS data, [we were] able to “OK-First has a quality severe weather training have staff aware of the potential which led to meeting program for Oklahoma emergency managers. Part every need of our citizens with shelters, meals, and of that includes a question and answer session with data for the daily repair crews for their safety during the media. This allows a connection between the recovery operations” (Jerome McCalvin, fire chief, governmental employees and those who report on Marlow Fire Department). TV. It is a useful and open dialogue allowing a better understanding of how each entity works. Great “How can a person single out any one event? It’s EV- program!” (Jed Castles, meteorologist, KWTV News ERY event, including not only the high-end tornado 9, Oklahoma City, Oklahoma).

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2021 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC Table 4. A comparison between different aspects of the OK-First program in 1997 and 2017. OK-First in 1997 OK-First in 2017 Training Free classes Free classes 8-day certification class 4-day certification class 2.5 h of required online prerequisites for certification and No class prerequisites assistant certification classes Class materials printed and provided in 3-in. binder Class materials provided via electronic book One certification level Two certification levels: Certified and assistant certified Certification and assistant certification test with ≥70% No formal testing passing grade required Recertification training required for certified participants Recertification training required for certified participants only (18 months) (18 months) and assistant certified participants (24 months) 2-day recertification class 1-day recertification class No online classes Asynchronous online recertification classes Instruction provided by Mesonet, NWS, and broadcast Instruction provided by Mesonet and NWS meteorologists meteorologists 50 people trained per year 400 people trained per year Technology Participants given free, password-protected access to high- Participants given free, password-protected access to high- quality weather and radar data products quality weather and radar data products User-customizable, mobile-compliant multihazard weather Multihazard weather data web page featuring plug-in tech- data web page featuring JavaScript-powered interactive nology (WxScope) for interactive maps maps Multipanel RadarFirst software for viewing radar data No RadarFirst software interactively (zoom, animate, and pan data) Select participants given a brand new desktop computer to take back to their jurisdiction (first three certification Participants not provided with a computer classes) Mobile weather apps (Mesonet and RadarScope app) given Mobile app technology did not exist to participants at classes Miscellaneous Feedback from users a core component of the program Feedback from users a core component of the program Classroom feedback provided via paper survey after every Classroom feedback provided via electronic survey after class every class Program direction formally guided by 15-member OK-First Advisory Committee, composed of 10 or more current Additional feedback obtained on ad hoc basis OK-First members and several nonmembers such as the NWS and FEMA (started 2011)

FINAL THOUGHTS. Over the past two decades, in Table 4. It is important to emphasize that a majority the Mesonet has successfully engaged, educated, of the changes to the program over its history have and empowered more than 1,400 public safety of- been user driven and provided through a variety of ficials across Oklahoma through its award-winning feedback mechanisms including class discussions, OK-First program (see sidebar for OK-First/Mesonet class evaluations, and the OK-First Advisory Com- awards). This has been accomplished by providing mittee. high-quality weather data tools, routine classes, and OK-First has shown that there is immense value continual follow-up support to its trained members. in providing nonscientific audiences with complex While the model of pairing data with training re- meteorological information when paired with well- mains the same 20 years later, many of the details of designed, relevant, and routine training. In fact, it is the program have evolved over time, as documented argued that officials should never be expected to be

2022 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC able to properly interpret meteorological information and make good decisions from the data without some OK-FIRST/MESONET AWARDS level of training. With the proliferation of weather • OEMA Outstanding Contributor to Emergency Man- information available through a variety of sources, it agement. Awarded to Oklahoma Climatological Survey is increasingly difficult to educate a growing base of Director Dr. Kenneth Crawford for his work with the users. Despite this, weather education programs such Mesonet and OK-First (1998). as OK-First, NWS storm spotter training, and Coop- • Innovations in American Government finalist (1999). erative Program for Operational Meteorology, Educa- • OEMA Outstanding Contributor to Emergency Manage- ment. Awarded to OK-First Program Manager Mr. Dale tion, and Training (COMET; Johnson et al. 2015) have Morris for his work with OK-First and public safety demonstrated that it can be done for specific audiences. officials (1999). Looking toward the future of OK-First, program • American Meteorological Society Special Award. Given staff are evaluating the feasibility of adding displaced to several agencies and television stations for outstand- real-time capability to the OK-First Weather Briefing ing and well-coordinated actions before, during, and page and RadarFirst software to support weather after the historic 3 May 1999 tornado outbreak in central event laboratory exercises with more realistic deci- Oklahoma (2001). • Stockholm Challenge finalist. This award recognizes sion-making time frames. In terms of training mate- programs that accelerate the use of information tech- rials, there are plans to build an online repository of nology for the social and economic benefit of citizens current OK-First educational materials (e.g., recorded and communities and is offered by Stockholm, Sweden presentations, screencast tutorials, quizzes, cheat (2001). sheets, and laboratory exercises) for informal, self- • Innovations in American Government Award. This rec- paced study outside the learning management system ognition is given to the five most innovative government environment. Furthermore, and most importantly, programs and is awarded by Harvard University’s John F. Kennedy School of Government (2001). the program will continue listening to the feedback of • American Meteorological Society Special Award. For its participants and advisory committee to best guide serving Oklahoma and the meteorological community future changes to training and data tools. by providing high-quality data and products (2005). As the suite of NWS products and services con- • OEMA Outstanding Contributor to Emergency Man- tinues to evolve over time, especially in the area of agement. Awarded to OK-First Program Managers Mr. decision-support services for the public safety com- Deke Arndt and Mr. Andrew Reader for their work with munity, it is imperative that training be an included OK-First and public safety officials (2009). component. As has been learned through OK-First, training not only provides a forum for receiving critical feedback that can result in improved tools past and present for its annual support of the Oklahoma and class content, but it also facilitates relation- Mesonet and our outreach programs. ship building and collaboration and promotes open communications between the different elements of the integrated warning team (NWS, public safety, REFERENCES and media). It is these relationships, ultimately, that Brock, F. V., K. C. Crawford, R. L. Elliott, G. W. Cuperus, dictate the local preparation for and effective manage- S. J. Stadler, H. L. Johnson, and M. D. Eilts, 1995: ment of high-impact weather events. The Oklahoma Mesonet: A technical overview. J. Atmos. Oceanic Technol., 12, 5–19, https://doi ACKNOWLEDGMENTS. The Oklahoma Mesonet .org/10.1175/1520-0426(1995)012<0005:TOMATO would like to thank the more than 1,400 public safety >2.0.CO;2. officials who have been a part of OK-First. Thank you for Carlson, J. D., and T. G. Bidwell, 2008: OK-FIRE: A new your commitment to the program, your many sacrifices to weather-based decision support system for wildland your communities, and the life-saving decisions you make fire managers. Oklahoma State University Coopera- every year. To the past program managers of OK-First— tive Extension Service Rep. L-326p Dale Morris, Deke Arndt, Andrew Reader, and Nicole Changnon, S. A., K. E. Kunkel, and B. C. Reinke, 1996: Giuliano—and the many current and former full-time Impacts and responses to the 1995 heat wave: A call and student staff at the Mesonet that have supported the to action. Bull. Amer. Meteor. Soc., 77, 1497–1506, program, thank you for 20 amazing years. Your tireless https://doi.org/10.1175/1520-0477(1996)077<1497:IA efforts have been key to the success and longevity of the RTTH>2.0.CO;2. program. Finally, we would like to extend our gratitude to Doswell, C. A., III, A. R. Moller, and H. E. Brooks, Governor Mary Fallin as well as the Oklahoma legislature 1999: Storm spotting and public awareness since

AMERICAN METEOROLOGICAL SOCIETY OCTOBER 2018 | 2023 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC the first tornado forecasts of 1948. Wea. Forecast- Morris, D. A., K. C. Crawford, K. A. Kloesel, and J. M. ing, 14, 544–557, https://doi.org/10.1175/1520- Wolfinbarger, 2001: OK-FIRST: A meteorological 0434(1999)014<0544:SSAPAS>2.0.CO;2. information system for public safety. Bull. Amer. Me- FEMA, 2017: Disaster declarations by state/tribal teor. Soc., 82, 1911–1923, https://doi.org/10.1175/1520 government. Department of Homeland Security, -0477(2001)082<1911:OFAMIS>2.3.CO;2. accessed 15 May 2017, www.fema.gov/disasters/grid —, —, —, and G. Kitch, 2002: OK-FIRST: An /state-tribal-government. example of successful collaboration between the me- Johnson, V., R. Jeffries, G. Byrd, W. Schreiber-Abshire, teorological and emergency response communities E. Page, B. Muller, and T. Alberta, 2015: Celebrating on 3 May 1999. Wea. Forecasting, 17, 567–576, https:// COMET’s 25 years of providing innovative education doi.org/10.1175/1520-0434(2002)017<0567:OFAEOS and training. Bull. Amer. Meteor. Soc., 96, 2183–2194, >2.0.CO;2. https://doi.org/10.1175/BAMS-D-14-00276.1. NOAA, 1977: Johnstown Pennsylvania flash flood of Klockow, K. E., R. A. McPherson, and D. S. Sutter, 2010: July 19–20, 1977. Natural Disaster Survey Rep., 60 pp. On the economic nature of crop production decisions —, 1991: Shadyside, Ohio, flash floods, June 14, 1990. using the Oklahoma Mesonet. Wea. Climate Soc., 2, Natural Disaster Survey Rep., 127 pp. 224–236, https://doi.org/10.1175/2010WCAS1034.1. —, 1994: Southeastern United States Palm Sunday McPherson, R. A., and K. C. Crawford, 1996: The tornado outbreak of March 27, 1994. Natural Di- EARTHSTORM Project: Encouraging the use of real- saster Survey Rep., National Weather Service, 1325 time data from the Oklahoma Mesonet in K–12 class- East-West Highway, Silver Spring, MD, 20910-3283, rooms. Bull. Amer. Meteor. Soc., 77, 749–761, https://doi 75 pp. .org/10.1175/1520-0477(1996)077<0749:EA>2.0.CO;2. Ziolkowska, J. R., and Coauthors, 2017: Benefits —, and Coauthors, 2007: Statewide monitoring of the and beneficiaries of the Oklahoma Mesonet: A mesoscale environment: A technical update on the multisectoral ripple effect analysis. Wea. Climate Oklahoma Mesonet. J. Atmos. Oceanic Technol., 24, Soc., 9, 499–519, https://doi.org/10.1175/WCAS 301–321, https://doi.org/10.1175/JTECH1976.1. -D-16-0139.1.

2024 | OCTOBER 2018 Unauthenticated | Downloaded 10/05/21 05:52 PM UTC