U-TOPIA: A Ubiquitous Environment with a Wearable Platform, UFC and Its Security Infrastructure, pKASSO

Kyu Ho Park,Ki-Woong Park,Jupyung Lee,Jong-Woon Yoo, Seung-Ho Lim, and Hyun-Jin Choi

Computer Engineering Research Laboratory, School of Electrical Engineering and Science, Korea Advanced Institute of Science and Technology {kpark,woongbak,jplee,jwyoo,shlim,hjchoi}@core.kaist.ac.kr

Abstract. U-TOPIA, introduced in this paper, is an advanced ubiqui- tous computing environment mainly focusing on a university campus. Research in U-TOPIA spans various components each of which is es- sential to realize U-TOPIA: from user device hardware/software, , communication technology, indoor/outdoor testbed, middle- ware to practical applications and security infrastructure. We designed and implemented a wearable platform from the scratch and make use of it as a main user device inside U-TOPIA. In addition to this, as a new user interface, we developed a device, called i-Throw to communicate with U-TOPIA in an intuitive manner. For data communication and location tracking in U-TOPIA, campus-wide indoor and outdoor testbed was deployed. To keep up with secure and dynamic U-TOPIA environment, a new security infrastructure,called pKASSO, and extensible , μ-ware, was developed. Finally, as a prac- tical application for U-TOPIA, we implemented a ubiquitous testbed room where multiple users interact with various ubiquitous devices or other users securely in a user-friendly manner. Integrating these compo- nents all together, we show that U-TOPIA can be a realistic role model to improve current paradigm of environment one step forward within a few years.

1 Introduction

Ubiquitous andpervasivecomputinghavehadawide ranging influence in the ideas ofhow the future would look like.Ubiquitous computing, already described in the early 1990 byMark Weiser, can be seen todayinvarious forms [1,2,3,4]. In recent years, the rapidprogress ofubiquitous andpervasivecomputingtechnol- ogy, fueled by either a government-led oracompany-led efforts to encourage the research, has led to the emergence of various ‘smart’ places powered by ubiq- uitous computingtechnology, rangingfromsmartroomand smart campus to smart city.

M. M¨uhlh¨auser et al. (Eds.): AmI 2007 Workshops, CCIS 11, pp. 183–193, 2008. c Springer-Verlag Berlin Heidelberg 2008 184 K.H. Park et al.

 Testbed (Wi-Mesh [Outdoor] )  Testbed (ZigBee, UWB [Indoor] )

 pKASSO, Security  µ-ware, Middleware

UFC (Ubiquitous Fashionable Computer)

Fig. 1. Overall Architecture of U-TOPIA

In 2005, our team launched a government-funded project aimed at realizing a campus-wide advanced ubiquitous computingenvironmentuntil the end of 2007.The ubiquitous computingenvironment was named as U-TOPIA,where ‘U’ stands for ‘ubiquitous’ and ‘TOPIA’ stands for ‘place’inGreek.Inorder to realize U-TOPIA, wedeveloped a wearablecomputer that allowspeopleto exploitubiquitous computingenvironment in auser-friendly manner andse- curityinfrastructure for secure ubiquitous services based on our testbed and middleware called μ-ware.The most importantcontribution of U-TOPIA can be summarized as follows; –UFC:Awearablecomputer with modularity andextensibilityis imple- mented based on ARM9 processorand embedded . Our hardware platform iscalled Ubiquitous FashionableComputer (UFC), which is named based on our special emphasis on its wearability, aestheticdesign andclose interaction with ubiquitous environment. –i-Throw:Anew human computer interface called i-Throw is invented to throw/get information in one’s UFCto the other UFC(s)orObjects when athrowing/gettingaction istaken by hishands. – μ-ware: It iscomposed of light-weight service discovery protocol, distributed information sharing, contextmanager and instance service loader, all of which are useful to manage dynamicdataandto develop new application that utilizes various ubiquitous resources. – pKASSO: AnewPKI-based security mechanism and securityinfrastruc- ture, pKASSO,isdevised for secure andseamless transaction betweenUFC and U-TOPIA where various devices frequently interact with each other or asurrounding infrastructure. In addition to this, fordatacommunication and location tracking in U- TOPIA, campus-wide indoorand outdoor testbed was installed.These efforts are closely related with realizing U-TOPIA in our campus. U-TOPIA: A Ubiquitous Environment with a Wearable Platform 185

2 U-TOPIA Internal

Overall architecture of U-TOPIA is described in Fig.1.U-TOPIA consists of various components each of which is essential to realize U-TOPIA: from wear- ableplatform (UFC), user interface (i-Throw), middleware (μ-ware) and secu- rity server (pKASSO) to practical and secure applications, and indoor/outdoor testbed.Wehavetried to pusheachresearchfieldastepforward to meet the ambitious goal, realizing a campus-wide advanced ubiquitous computingenvi- ronment.

2.1 Indoor and Outdoor Testbed Since U-TOPIA aims foracampus-wide environment,it is essential to builda large-scale testbed inside which various services can be operated.Twoimpor- tantcomponents of a target testbed are a communication infrastructure anda location-tracking infrastructure. Communication infrastructure laysthegroundwork forubiquitous computing. Communication inside U-TOPIA ismadepossiblebywireless mesh network that is installed inside our campus.Location-tracking infrastructure is necessary for a location-based service.Inoutdoorenvironment,wemadeuseof widely-used GPS information to track the location ofmoving objects.Inindoorenvironment, webasically used ZigBee signal strength-based location trackingmechanism.To do this,we installed enough number ofZigBee sensor nodes inside two selected buildings inside U-TOPIA. During the measurement, however,wefound that the resolution of location sensingusingthismechanism was notsufficientfor our tar- get application. Thus,wealso utilized UWB-based location trackingdevice[13] whose typical accuracyis 6inches(15cm). Due to the high cost ofthissolution, UWB-based location trackingdevice have installed in only two rooms inside U-TOPIA.

2.2 Middleware, μ-Ware In U-TOPIA, we assume a situation where thousands ofusersmovehereand there,interact with each other orubiquitous computingenvironment, share in- formation with authorized other users, access to diverse devices fordiverse pur- poses, runvarious location-based applications.Inthissituation, an extensible middleware framework is necessary to keep up with highly variabledynamicen- vironment.Wehavebeenworking with middleware team andthey developed a extensiblemiddleware, called μ-ware[11]. μ-ware iscomposed of light-weight ser- vice discovery protocol, distributed information sharing, contextmanager and instance service loader, all of which are useful to manage dynamicdataand develop new application utilizing various ubiquitous resources.

2.3 Wearable Platform, UFC In U-TOPIA, ahardware platform is necessary to provide a user with plentiful ubiquitous computingresources.The hardware platform shouldbelight-weight, 186 K.H. Park et al.

Hub (inside) Hub (inside) (inside) Camera Storage (inside) PANDA (ZigBee) Battery (inside)

WLAN (inside) i-Throw Main module (LCD)

Fig. 2. UFC Platform Design and Implementation easy to carry, easy to use and itshouldhave aestheticappearance andsocial acceptance.Wechose to design and implementawearableplatform fromthe scratch andmakeuseof itasamain user device inside U-TOPIA. Our wearable platform,incontrast to either PC orhandhelddevice, allowsauser to carry the computingdevice in acomfortableand natural manner, because clothes havebeen already a essential component of our daily lives.Large surface area ofclothes can be utilized for various purposes andthus,I/O interface does nothaveto be located in only asmall-sized computingdevice.Moreover, the wearableplatform makes iteasier to measure and gather bio signal data such as temperature and heart rate byintegratingbody-attached sensors andcomputing devices upon asameclothes interface. The implemented UFCplatform isshown in Fig. 2.UFCmodules are dis- tributed on a garment, consideringthedistribution of weight and aestheticde- sign. Moreover, each UFCmodulecan be attached anddetachedeasily on a garment, allowingusersto construct one’s own UFCplatform.Since weutilized astandard USB protocol to communicate between the main moduleand various UFCmodules, due to the hotswap capabilityof USB devices, each UFCmodule can be attached anddetachedwhilethesystem isrunning. The success of wearablecomputer relies on not only wearability, but also the aestheticappearance andsocial acceptance.Wetried to findthesolution to fulfill the requirements by repeating the prototypingbodystormingprogresses.We defined the target users as younguniversity students anddrew design concepts by analyzingtheiractivities in everydaylife andfashion trend.Inaddition, wehave made effort for each part oftheUFCplatform tolook like familiar fashionable components: forexample, an attachable/detachablemodule iscomparableto a button ofclothingandani-Throw device iscomparableto aring.Also, PANDA can be worn as a form ofanecklace.

2.4 User Interface, i-Throw In U-TOPIA, since plentiful resources locate outside a user, rather thaninside a user device, abrand-new user interface isrequired to manipulate various outside U-TOPIA: A Ubiquitous Environment with a Wearable Platform 187

Fig. 3. Gesture Sets of i-Throw resources in auser-friendly manner.Tomeet thisrequirement,wedeveloped a wireless gesture recognition device, called i-Throw. Usingthisdevice, ausercan express one’s intention easily by using one’sspatial movementandhandgesture. It issmall enough to be wornonone’sfinger like a ringandhasathree-axis andathree-axismagneto-resistivesensorforrecognizingagesture andthedirection ofthefinger.Italso has a ZigBee transceiver fortransmitting the recognized gesture information to the UFCplatform. The gesture recognition consists oftwo stages: feature extraction stage and testing stage.The feature extraction stage isapreprocessing stage to findref- erence features ofeachgesture.Afeature f isrepresented by afour dimensional vectorasfollows: f =(ATHx,ATHy,ATHz,TH ), (1) where ATHx, ATHy and ATHz are the acceleration thresholds ofeachaxisand TH istime duration threshold.Inthe feature extraction stage,weshouldfind appropriate thresholds foreachpossible input gesture.Due to the limitation of space,we omit the detailed explanation of the feature extraction stage.Inthe testing stage,i-Throw device compares the output of the accelerometer with each reference feature for over TH seconds.If one of the features ismatched, theni-Throw transmits the recognized gesture to the UFCplatform viaZigBee interface.The gesture recognition isdesigned to be simpleenough to runonamicro-controller inside the i-Throw by extractingtheminimum set ofrequired features andusingthreshold-based simplefeatures.We summarized and illustrated the gesture sets that the i-Throw recognizes in Fig.3.Other possiblegestures, scrollingup/down andcanceling, are intentionally omitted here. Every time a UFCuserpoints to adevice, the UFCplatform displaysthe selected target device upon its screen. This feedback information helps the UFC user findthecorrect target device.Similarly, ascanninggestureallows the user toinvestigate controllabledevices inside the room.Thisscanning operation is 188 K.H. Park et al.

KIOSK pKASSO

User A User B Authentication Latency of pKASSO: min. 0.082 sec Authentication Latency of conventional PKI : min. 5.02 sec

Fig. 4. Authentication flow description with pKASSO similar to the operation ofmovingamouse pointer across severalicons in a typicalPCdesktopenvironment.‘Ready-to-receive’ gesture is necessary fora UFCuserto express one’s intention to receive other UFCusers’objects.When oneusermakesapointing orscanninggesture,onlylimited users who makes the ‘ready-to-receive’ gesture can be selected.

2.5 Security Infrastructure, pKASSO

In order to provide a full-fledged security solution especially tailored for U- TOPIA, wherein numerous devices andsensors with severe resource-constraints interact with each other,wedeveloped an computationally efficient PKI-based securityinfrastructure, pKASSOenhanced with singlesign-on anddelegation technology. Itenables a cost-effective but uncompromisingly secure development of UFC.The delegation mechanism ofpKASSOmakesitpossibletooffloads complex cryptographyoperationsfrom UFCto server-side so that itsignifi- cantly improves authentication latency as well. Accordingto the performance evaluation, the authentication latency(Avg.0.082sec)ismuchshorter than a contact type smart card (Avg. 4.31sec) andaconventionalPKI-based authen- tication latency(Avg.5.01sec)[5].Overall process of authentication based on pKASSO is illustrated in Fig.4.

1. User A sends a challenge message to user Bwhowants to communicate with user A. 2.User B generates an authentication request message (two symmetrickey operations) andsends itto pKASSO. and itperforms transactionsfor veri- fication and authentication on behalf ofuserB. U-TOPIA: A Ubiquitous Environment with a Wearable Platform 189

Folder Bookcase U-Trash U-Printer

E-mail Mail Box

U-Kiosk

Schedule Clock UFC User 1

UFC User 2

U-Display

Fig. 5. The Concept of the Ubiquitous Testbed Room

3. pKASSO makes a response message andtransmits itto user A. 4.The authentication iscompleted with the arrivalofaconfirming message fromuserB.

3 Target Application: User-Friendly Interaction with U-TOPIA

Asmentioned in Section 2.4,wetook ‘user-friendly interaction with ubiquitous devices using i-Throw’ as a target application. To execute thisapplication, wehave implemented a ubiquitous testbed room where multiple UFCusersinteract with various ubiquitous devices or other UFCusers.Fig. 5 illustrates the concept ofthe ubiquitous testbed room which presentapractical application that runsupon the UFCplatform and the ubiquitous devices in testbed,which makes itpossibleto exchange the various objects andcontrol ubiquitous devices very easily.

3.1 Motivation

Due toits small form factor, most portabledevices,including our UFCplatform, have only small-sized display and limited input devices.The UFCplatform has 2.5” LCD display and 12 input buttons,which are definitely insufficientto mon- itor the status ofaUFCmain moduleand various modules, control the modules, andsendauser’s intention to the UFCplatform. Thisproblem isexacerbated when a UFCusertries to control various ubiqui- tous devices using one’s UFCplatform: as the number ofcontrollableubiquitous devices increases,itbecomes more inconvenientto find oneamongthemand exchange information with it, due to the small-sized display and limited input devices oftheUFCplatform. 190 K.H. Park et al.

We attempt to resolvethisproblem by makingfull use ofspatial resources inside the testbed room: given that various ubiquitous devices are spatially dis- tributed inside the testbed room, a UFCusercan express one’s intention easily by using one’sspatial movementandgesture.Forexample,let us assume that one UFCusertakesapicture and intends to put it on a publicdisplay so that other peoplecan see the picture he takes.From the perspective of the user, the most naturalwayofreflecting one’s intention on the environment ispointinghis finger at the publicdisplay andthrowing one’spicture at the publicdisplay. If thiskind ofauser-friendly spatial gesture interface is supported, the limitation oftheI/Oresources oftheUFCplatform can be overcome by fully utilizing abundantspatial resources. FortheUFCplatform to support such a practical and secure interface, the followingcomponents are necessary:

– gesture recognition device recognizes the target device that a UFCuser ispointingatand the gesture such as ‘throwing’ and ‘receiving’. – location tracking device keeps track ofaUFCuser’s location. This is necessary because finding the target device that a UFCuserispointingatis dependent on the absolute location oftheUFCuser.Weutilized UWB-based location trackingdevice[13] whose typical accuracyis 6inches(15cm). –locationservergathers andmanages the location information ofboth UFC users andubiquitous devices.Whenone UFCuserpoints at a specific device, the recognized gesture information issentto the location server and itfinally decides what the target device is. – Mutual Authentication and secure key distribution:Inorder to pro- vide secure andseamless transaction betweenUFCand U-TOPIA where various devices frequently interact with each other orasurrounding infras- tructure, pKASSO isdeployed. – Application that runs upon UFC platform infers a UFCuser’s inten- tion based on agesture, a target device andprevious operationsandconducts acorresponding operation.

3.2 Detection of Target Device

Our target detection algorithm isbasedon Coneselection which isusedin virtual computingenvironments[9]. A cone iscastfrom i-Throw andasetofdevices that intersect with itarechosen. Additionally, wehavemodified that typical coneselection algorithm tovary the area ofthecone adaptively toimprove the overall target detection accuracy. To do this, the orientation of i-Throw andtheposition oftheUFCuseranddevices shouldbeknown. The location information is gathered andmanaged by virtual map of μ-ware.The virtual map contains interactive objects aroundauserin aspecific ubiquitous environment and supports several services as follows:

– Interactive Object Registration: Forfastspatial queries, the repository of virtual map maintains a currentsnapshot oftheinteractive objects. U-TOPIA: A Ubiquitous Environment with a Wearable Platform 191

ID : UFC2 ID : TV_1 Interactive object Location : RoomA( 500, 200 ) Location : RoomA( 30, 200 ) Interaction : UbiSpace Interaction : UPnP Command Mapping Command Mapping Throwing → File transfer Throwing → Turn on Hands-up → Set status to ready-to → Rolling right volume up -receive ….

TV UFC2 Throwing after pointing

UFC1 Printer News-kiosk

File transfer UFC2 TV

Printer KIOSK UFC1

Fig. 6. Virtual map and interactive objects

– Interactive Object Discovery: WhenUFC detects logical address change, the virtual map repositoryisresolved bylogical address key. – Target Selection in a Virtual Map: UFCcan see the target in the given boundary space with objects with priorities. The scope ofthevirtual map isautomatically adjusted upon user’s loca- tion(inside building or outdoors). In the virtual map, each interactive object is tagged with geographic information and attributes tointeract with.Anin- teractive object isan abstraction unit ofphysicalobject on the virtual map. Interactive object contains location, andservice attributes tointeract with.Fig- ure6 showsthevirtual map and interactive objects.InFigure 6, the UFC1 has information about UFC2’s location, interaction method, andthecommandcon- figuration that assignsthrowingmotion as transferringafile.Andtheorientation of i-Throw can be obtained by combining the accelerometer andmagneticsensor outputs.The accelerometer isusedfortiltcompensation. By usingtheorienta- tion andposition information, target detection can be performed properly. Wedefine the target device sets which consist ofpossible target devices in the ubiquitous testbed and the characteristics ofeachone,which are summarized in Table 1. Among these devices, the news kiosk automatically gathers recent news

Table 1. Target Device Sets

Target device Supported objects Flow of objects U-Display News, Photo Input, Output U-Kiosk News Output U-printer News,Photo Input U-trash News,Photo,music Input Clock, Mail, Bookcase Supported Application Input, Output UFC news,photo,music Input, Output 192 K.H. Park et al. from internet web sites anddisplayseachone, that is refreshed every10seconds. Whenone UFC user sees the interesting newsupon the newskiosk, he orshecan obtain the newsby making ‘receiving’ gesture towards the newskiosk.The u-trash functionsasasymbol of ‘deletingafile’. Similarly to the naturalwayofusingan ac- tual trash, throwingsomethingthatisuseless anymore at the trash,if one UFCuser makes a throwinggesturetowards the u-trash, the current object will be deleted automatically. The u-trash device isdifferentfrom other devices in that it is not an electricdevice;itactsonly as a marker standingforaparticular operation and thus actualoperation is notconducted inside it.Thisexamplegives us insight as to how to fully utilize spatial resources inside the room.If various markers whose symbolicmeaningcan be easily interpreted are added inside the roomandeachcor- responding operation isefficiently conducted, the spatial gesture interface allows UFCusersto conduct various operations in auser-friendly manner.

4Conclusion

U-TOPIA, introduced in thispaper,is a campus-wide advanced ubiquitous computingenvironment.We summarized five essential components to realize U-TOPIA paradigm in our campus:indoor/outdoor testbed, middleware (μ- ware), wearableplatform (UFC), user interface (i-Throw), securityinfrastruc- ture (pKASSO). Since 2005, wehavetried to push each research fieldastepfor- ward to meet the ambitious goal, realizing a campus-wide advanced ubiquitous computingenvironment.Specifically, wedesigned and implemented a wearable platform fromthescratchandmakeuseof itasamain user device inside U- TOPIA. In addition to this, as a new user interface,wedeveloped a wireless gesture recognition device, called i-Throw. To keep up with secure anddynamic U-TOPIA environment, a new securityinfrastructure,called pKASSO, andex- tensiblemiddleware, μ-ware,was developed. In addition to this, fordatacommunication and location tracking in U- TOPIA, campus-wide indoorand outdoor testbed was installed.Finally, as a practical application for U-TOPIA, we implemented a ubiquitous testbed room where multipleusersinteract with various ubiquitous devices or other users in auser-friendly manner.Allthese efforts are closely related with realizing U- TOPIA in our campus.WebelievethatU-TOPIA can be a realisticrolemodel toimprove current paradigm ofubiquitous computingenvironment within afew years.Asafuturework,we will attempt to apply our indoorapplication to outdoor testbed, so thatausercan extract suitable information from outdoor components, such as building, billboard,large display and vehicles.

References

1. Mann, S.: Wearable computing as means for personal empowerment. In: Proc. 3rd Int. Conf. on Wearable Computing (May 1998) 2. Pentland, A.: Wearable intelligence. Proc. Scientific American 1, 276 (1998) U-TOPIA: A Ubiquitous Environment with a Wearable Platform 193

3. Starner, T.: The challenges of wearable computing: Part 1. In: Proc. IEEE Micro, July 2001, pp. 44–52 (2001) 4. Weiser, M.: The computer for the 21st century. In: Proc. Scientific American, September 1991, vol. 3, pp. 66–75 (1991) 5. Park, K.W., Seok, H.C., Park, K.H.: pKASSO: Towards Seamless Authentication providing Non-Repudiation on Resource-Constrained Devices. In: Proc. 21st IEEE Symposium on Pervasive Computing and Ad Hoc Communications, May 2007, vol. 2, pp. 105–112 (2007) 6. Lee, J., Lim, S.H., Yoo, J.W., Park, K.W., Choi, H.J., Park, K.H.: A Ubiquitous Fashionable Computer with an i-Throw Device on a Location-Based Service En- vironment. In: Proc. 21st IEEE Symposium on Pervasive Computing and Ad Hoc Communications, May 2007, vol. 2, pp. 59–65 (2007) 7. IEEE802.15.2 Specification, Coexistence of Wireless Personal Area Networks with Wireless Devices Operating in Unlicensed Frequency Bands (August 28, 2003) 8. Back, M., Lahlow, S., Ballagas, R., Letsithichai, S., Inagaki, M., Horikira, K., Huang, J.: Usable Ubiquitous Computing in Next-genration Conference Rooms: Design, Evaluation, and Architecture. In: Proc. UbiComp 2006 Workshop, Septem- ber (2006) 9. Liang, J., Green, M.: JDCAD: A Highly Interactive 3D Modeling . Proc. & Graphics 4, 499–506 (1994) 10. Kang, M., Chong, J., Hyun, H., Kim, S., Jung, B., Sung, D.: Adaptive Interference- Aware Multi-Channel Clustering Algorithm in a ZigBee Network in the presence of WLAN Interference. In: Proc. IEEE International Symposium on Wireless Per- vasive Computing, Puerto Rice (February 2007) 11. Song, Y., Moon, S., Shim, G., Park, D.: μ-ware: A Middleware Framework for Wearable Computer and Ubiquitous Computing Environment. In: Proc. 5th Con- ference on Pervasive Computing & Communications(PerCom 2007) (March 2007) 12. UPnP Forum, UPnP Device Architecture 1.0, Version 1.0.1 (December 2003) 13. Ubisense, http://www.ubisense.net 14. KAIST UFC Project, http://core.kaist.ac.kr/UFC