2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

Avatar Driven VR Society Trends in Japan

Liudmila Bredikhina* Takayuki Kameoka† Shogo Shimbo‡ Akihiko Shirai§ Geneva University The University of Electro-Communications Waseda University GREE, Inc.

ABSTRACT application of new methods of production or sales; 3) opening a This paper reports recent development, experiences and evaluations new market; 4) acquiring of new sources of supply; and 5) new of social VR platforms in Japan that have been motivated by VTubers industry structure such as creation or destruction of a monopoly and VRChat since 2018, especially the “Virtual Market”. Virtual position [22, p.90]. Innovation process is often described as being Market was emulated from the conventional ‘comic market’ model “contingent” and Pavitt [15] proposes a framework to understand- which drives “Doujinshi”, fanzine creators. The third edition of ing innovation process based on the “know-how” (improvement of Virtual Market has been visited by 700,000 visitors. This paper also products, processes or services and a change in market demand) reports six cases of “VRSionUp”, which include hybrid workshops and on the involvement of “learning through either experimentation in both physical locations and VR. As another case, we introduce (trial and error) or improved understanding (theory)” [15, p.88]. He international live performance between Australia and Japan by “Vir- then proceeds to explain that innovation can be divided into three tual Cast” in SIGGRAPH ASIA 2019 Real-Time Live. As the latest overlapping processes: 1) the production of scientific and techno- challenge of pure virtual academic conference, we will also intro- logical knowledge; 2) the translation of knowledge into working duce “Virtual Gakkai”, an academic conference with poster and artifacts; 3) responding to and influencing market demand [15, p.88]. keynote sessions. The following paragraph will explore VR evolution based on these theoretical frameworks. Index Terms: Virtual conferences, VR platforms, multi-site events, In the last decades, VR went through four stages. VR1.0, the first massive community management, avatar society, VTuber version, is represented by the “Sword of Damocles” also known to be the first VR head-mounted (HMD) system that was only available 1MOTIVATION: “VR4.0”-DEVELOPMENT OF AVATAR- in Sutherland’s Lab and presented in 1968 [23]. The first VR craze DRIVEN VR SOCIETY came in 1990’s [23] and VR2.0 saw an improvement in the technol- In the current era of self-communication, technology has become so- ogy with graphic workstations (GWS), purchasable HMD and data ciety [1] and is constantly influencing and generating various forms globes. However, the VR developmental environment was limited to of social engagements [14] beyond the physical world. The Web laboratories with significant budgets. International collegiate Virtual has transformed the nature of our daily activities and has become an Reality Contest (IVRC) was founded in 1993 by pioneer researchers integral part of our daily lives. Virtual reality (VR) enables users to of Japanese virtual reality in an academic setting. embody 3D virtual characters (in this paper referred to as “avatars”) IVRC has been incubating young engineers and researchers for with which they create content on YouTube, stream their encounters many years [3, 20]; however, it is limited to location-based VR. in Virtual Cast, interact with other users in VRChat. Compared Participants held interest in achieving a sense of reality via display to a 2D image or a pseudonym, 3D virtual characters allow users hardware such as haptic devices. It contributed to finding young to be physically embodied in a VR space and provide support for talented VR researchers in Japanese engineering schools. However, individual expression, resulting in ubiquity [8]. In recent years, most creations were too early for VR consumer market. VR2.0 avatar society in Japan has developed various socially participatory was based on local-area networks (LAN) or offline real-time graphic platforms. Driven by the avatar society, individuals have begun or- experiences. Most well-known creations were conceived in the game ganizing conferences, exhibitions and academic meetings in hybrid industry in the early 21st century. formats with participation in both physical locations and VR. Hereafter mobile technology and a proliferation of contents have Avatars connect users between each other on various network developed, making VR3.0 the age of VR democratization, and in- [2, 12] , generating social interactions in VR [11]. Virtual characters novations carried out with a commercial purpose [5]. VR3.0 was “provide access points in creation of identity and social life. Bodies developed by game engineers and by an interest in personal HMDs. people use in those spaces provide means to live digitally – to fully Individuals, researchers and developers began asking why VR is inhabit the world. It is not simply that users exist as just “mind”, but needed and what it can bring. VR4.0 is a human ecosystem where instead construct their identities through avatars” [24, p.40]. Virtual people build commerce, blockchains, AI and social networks that go worlds provide individuals with a clean state, an empty identity, beyond current social networks [18]. Japanese avatar driven society is the force1 behind the new industry structure and proliferation onto which users can ascribe any set of characteristics according 2 to their desires or content [4]. As people began creating avatars of applications for interacting in VR environments without HMD and engaging in community-based interactions in VR, different while cohabiting real and virtual worlds. broadcasting platforms emerged. This paper focuses on avatar driven VR society that is based on 2VTUBER TO AVATARS innovations and developments. Those two aspects are divided by The word avatar came from the Urdu “avatar”, Sanskrit “ava-tara”,¯ Schumpeter [17] into five types: 1) launch of a new product; 2) meaning God’s descent from heaven. In computer history, video games traditionally used “main character” for representing the player *e-mail: [email protected] as a vehicle, a fighter or a human. In 1979, video game “Avatar” was †e-mail: [email protected] the first game to use an avatar. In 1993, “Habitat” by Fujitsu and ‡e-mail: [email protected] §e-mail: [email protected] 1VTuber x Technology: What Is a VTuber(https://medium.com/@vrstudiolab/ vtuber-x-technology-what-is-a-vtuber-368c9efced50) 2IKINEMA launches Orion 2.0(https://cgsociety.org/news/ article/4450/ikinema-launches-orion-2-dot-0)

978-1-7281-6532-5/20/$31.00 ©2020 IEEE 497 DOI 10.1109/VRW50115.2020.00100 Lucasfilm Games was the first game that had habitable avatars in a content creation and sharing are democratized as creators are no town. “Worlds Chat” system (1997) is the first avatar chat system in longer limited to exposing their real faces and identities. 3D space. Current popular Avatars in VRChat are inspired by Virtual YouTu- 3.2 VRChat ber (VTuber) of Japan. VTubers are streamers and vloggers who VRChat (https://vrchat.com/) is a popular online virtual meet- use 3D computer generated (CG) virtual characters [10]. Originally ing platform that hosts and enables interactions. It lets users create, from Japan, VTuber tend to adopt anime like visual characteristics. publish, explore virtual worlds, socialize, attend events, take classes, VTubers use hardware and software that capture body motion, facial play games, watch movies, brainstorm concept and much more. As expressions, virtual clothes movements and lip-synchs users voice to if they were in the same room, users interact through speech, emotes, virtual character’s mouth — all in real-time rendering. The system emoji and body gestures with participants from all over the globe. consists on VR3.0 hardware, such as consumer HMD like HTC In VRChat there are public worlds that anybody can access at any Vive and Vive trackers, and game engines like Unity and/or Unreal time, as well as private worlds that participants create for themselves Engine. or for their gatherings with friends. Private rooms can also be used VTubers broadcast their content via mobile, VR devices or PC pro- for exhibitions, conferences or academic meetings. According to grams like OBS (Open Broadcaster Software) Studio. They can be VRChat guidelines, users can update their worlds as often as they individuals who act out on their own or they can be virtual characters like once they are accepted by VRChat. However, there’s a weight with specific backstories commissioned by companies. In the case restriction of world sizes for better performance. There are also of individuals acting out on their own, VTubers identities emerge technical specifications regarding shaders, mirrors and real time from desires to express themselves, have fun, socialize, embody and lights. Increasing number of participants would make the world interact through a 3D virtual character. Identities and interactions are heavier, causing issues. It is not recommended to construct a flat always positioned, as a way to understand the behaviour of others world without any obstructions: they function as spatial limitations and vice versa [25]; are based on context and content [16]. People’s that reduce drawing load. Creators can stream their play on various virtual social participations and interactions are mediated by those platforms such as Twitch and YouTube. VRChat can be played virtual identities [13, 26]. With the proliferation of human ubiquity as a single player, player versus player (PvP), cooperative game- and avatars, different companies responded to the growing market play (co-op), or as a cross-platform multiplayer. The game can be demand for virtual character communication and proposed platforms accessed in desktop mode or in VR; HMD is required for a more that will be discussed in the following section. immersive experience and greater control over avatar’s movements. 3CURRENT AVATAR PLATFORMS The play area can be seated, standing or room scaled. Supported VR HMD are: Valve Index, HTC Vive, Oculus Rift and Rift S, Win- The following section provides an overview of current platforms for dows Mixed Reality. In VRChat users can embody a pre-configured broadcasting avatar society. It also aims to showcase Japanese VTu- avatar or upload a personalized 3D CG model. Currently, in VRChat bers influence in VR 4.0. Each of the following platforms provides there’s a proliferation of avatars inspired by Japanese VTuber visual users with possibilities to create social environments, hold events or characteristics. participate in group gatherings. The proposed selection includes VR platforms, cross-platforms and mobile applications for interacting 3.3 Virtual Cast in VR environments with or without HMD. Interactions between 5 audience and members in virtual spaces, examples of support of Virtual Cast is a Japanese online VR live communication platform. multi-site events, hybrid formats with participation in both physical It can be played as a single player or as a massive multiplayer online locations and VR, interactions between physical and virtual venues game (MMO). The play area can be seated, standing or room scaled. and lastly interactions between audience members in virtual spaces Supported VR HMD are: Valve Index, HTC Vive, Oculus Rift and will be outlined. Rift S, Windows Mixed Reality. In Virtual Cast users can transform themselves into virtual characters by choosing pre-existing models 3.1 YouTube or by uploading their own 3D CG avatar. Virtual Cast supports full YouTube is commonly used by VTubers to broadcast their live events body interactions, eye and lip tracking via Vive Trackers. The world created on other platforms. YouTube chat allows the audience to is divided into various pre-configured rooms in which users can interact between themselves as well as with the creators of the interact. It is also possible to create personalized rooms. By sharing program. Although the interactions can only take place during a the Studio ID with other users, hosts can allow others to join their live broadcast, users who are in a different time zone can review the room before the broadcast begins. program even after the broadcast has ended (as the video stays up on Virtual Cast has a three-level licence system: consumer, business host’s channel). Unlike other platforms, YouTube provides a tool for and professional. In the case of the professional license, 7 people tackling language barriers: creators can add subtitles in any language. or more can simultaneously join a designated room. Virtual Cast However, they cannot be added in a live broadcast and creators can users are free to create their own programs such as talk shows only add them later on. YouTube is an outlet for expression where and conferences; or they can play games, participate in singing people can create content with virtual characters that would not competition. Participants interact with each other through voice and be transmitted on TV. Ads and Superchat, “Nagesen” (tossed coin; various 3D CG objects. They can use the ones incorporated in the micro funding by followers) are a new way of rewarding creators app, such as a camera to broadcast their show as if they were filming for their sustainable video production. In Japan there is a culture it in real life, a mirror or a crayon. The room can be streamed on of sharing latest visual art creations on video sharing websites like NicoNico Live, YouTube or SHOWROOM with the audience either YouTube or NicoNico. In 2018, Hatsune Miku3 (2007-) was the most interacting through written comments and questions or joining the famous virtual artist [6] whose original music videos were shared by hosts in their rooms (if invited). In the case of comments submitted users4 in a peer production synergy [9]. With 3D avatar technologies, to NicoNico Live or Twitter, they appear as 3D objects in hosts’ rooms. Viewers can also send gifts to the Virtual Cast creators if 3Hatsune Miku, translated as ‘the first sound of the future’ first appeared the content is viewed from and streamed on NicoNico Live. The 3D in 2007 as a vocal synthesized software created by Crypton Future Media, comments can be caught and used, creating real-time interactions employing the Yamaha’s software engine called Vocaloid between creators and their audience. 4Mikuu original song tag(https://ch.nicovideo.jp/ miraipachirisu/blomaga/ar1400394) 5SHOWROOM(https://corp.virtualcast.jp/)

498 Virtual Cast and its collaborators promote a common file format addon function “Emomo”10 that allowed players to create 3D car- VRM. It is based on a standard 3D format glTF2.0, a single file can toonized avatars that are overlayered on the mobile game screen express one avatar which includes mesh, bone, texture, shaders, face capture. blend-shape, spring bones for soft-body, hair and cloth simulations. VRM also founded VRM Consortium6 and various VRM supported 3.7 Cluster tools are currently being developed. Originally Cluster11 (2015-) is a massive easily accessible VR con- 3.4 SHOWROOM ference platform. It is possible to participate in a Cluster event from 7 Windows or Mac personal computers, or, for a more immersive SHOWROOM is a Japanese live streaming service commonly used experience, VR headset is advised. The platform creates a space for free daily live streams. It can be broadcast-ed from a smartphone, where hosts and audience can meet together in a 3D environment. personal computer or Virtual Cast. Hosts can receive direct support They are symmetrically positioned in a Cluster room for massive from their fans. The website is divided between several categories: audience optimization and interaction. By inhabiting 3D CG avatars, Men’s, idol, talent model, music, voice actors and anime, comedians creators interact with their audience with their voice; they can also or talk show, virtual, karaoke and non-professionals. Audience is use PDF slides, pictures and emotes. Audience can engage using asked to enter a desired room on the proposed website menu and emotes like clap or like. Recently, this platform’s business has been are taken to a page where a screen broadcasts the selected program. motivated by virtual talents. Kaguya Luna12 organized the first live During live streams, viewers can offer creators virtual gift items and music event in Cluster with the performance being simultaneously interact with them through written commentaries; viewers can also broadcast-ed in several cinema complexes. interact between themselves in the chat menu. They are part of the viewed content as SHOWROOM requires an avatar that is shown on 4 VRCHAT the “room’s” web page8. As of 2018, the enterprise has made $USD 41M. 4.1 VRChat user formation, from Mr. Shimbo’s survey Shogo Shinbo conducted a research on 576 users of VRChat13, 3.5 REALITY focusing on the Japanese community and the economic sphere. The REALITY [19] is “For avatar only” live entertainment platform research was divided into four parts: VRChat user trends, open created by Wright Flyer Live Entertainment (Subsider of GREE, communities, closed communities and VRChat economics. The user Inc) and available on Japanese AppStore or Google Play. Its slogan trends section explored VRChat users’ occupations: 28.% students, is “REALITY: Be yourself you want to be” and caters to people who 22.0% clerical workers and 16.8% communication engineers. 30% desire for a 3D avatar (VTuber) and want to communicate with their were invited to VRChat by friends and 40.5% became interested audience without other hardware or software requirements than a in VRChat after watching VTuber content. More than half of the smartphone. Through the use of iPhone’s “TrueDepth” or Android’s people who answered the survey spend 3 or more hours in VRChat computer vision-based face/head capture, REALITY performs high and 38.5% use VRChat daily, with the greatest number of loggins frame rate real time facial tracking. The application renders user’s between 8pm and 4 am. Most of users’ engagement in VRChat live movements and eye blinks onto the avatar. Rendering runs high revolved around visiting different worlds, communicating with other frame rate in real time, however, shading and hair simulations are avatars about daily life activities or talking about issues related animatic. to Unity and Blender. Other activities included eating, sleeping The app has a variety of hairstyles, eye colour, skin colour, gender, and “Game World”. VRChat is a place of freedom and personal and props to choose from. From a smartphone, REALITY motion expression, 63.8% of users found the appeal of VRChat in avatar capture technology enables users to connect with their audience and exchanges and 57.3% were interested in VR communications. to participate in collaborative streams with other avatars. REALITY Open VRChat communities can be easily observed from the out- is revolved around VTubers live streaming entertainment, chatting, side and the admission is more flexible, where’s closed communities avatar clothing, live game by avatars, and audience reactions and have restricted admissions (on invites) and are difficult to study. By virtual gifting that can be monetized. Several official programs with choosing the admission limit to a VRChat world, users can choose VTubers as hosts have been created on this platform, aiming primar- the number of people they want to interact with. The most com- ily at forms of entertainment such as: singing contests, Christmas mon choice was a world where only friends of the user who is in specials, gaming, etc. In some cases, the broadcast is also streamed the world could enter. Users would generally join each other and in real-time on YouTube. As the broadcasting can be achieved gather in a designated world. 24.3% of users were mostly active through the use of a smartphone, it does not require VR equipment in their own world, but Fantasy “Shukai jou” (Meetup space) and and PC. Tutorial world were at 28.6% and 22.8% respectively. Users would visit public worlds in order to make friends, at 47.2%, followed by 3.6 Mirrativ the desire to stroll around and change their minds. 31.1% made Mirrativ9 is a Google Play and Apple Store smartphone app for live friends by exchanging in public worlds (31.1%), being introduced streaming, which can be performed anytime and anywhere. Mirrativ to by friends of friends, and having exchanges at events held in aims to create a new type of social interactions by streaming mobile VRChat (27%). 80.6% of the users from this survey participated device’s content: playing videos, gaming, etc. The app is designed in large-scale events such a Virtual Market, which will be further to capture directly the mobile phone’s screen as the service was orig- discussed in the next section, and 70% participated in any kind of inally designed for mobile game broadcasting with player’s voice. VRChat events. Twitter is used to share photos taken in VRChat, Viewers interact with the creator through commentary. Mirrativ also expand friendships and gather information. 59.5% of users did not offers the possibility to connect to its service via an avatar created separate their Twitter and VRChat accounts and 57.1% acted the in the smartphone version of VRoid program, or via a Virtual Cast same way on both platforms. When studying closed communities, account, and broadcast live content. In 2018, Mirrativ released an 10Emomo avatar function(https://prtimes.jp/main/html/rd/p/ 6VRM Consortium (https://vrm-consortium.org/) 000000016.000033025.html) 7 SHOWROOM(https://www.showroom-live.com/) 11Cluster(https://corp.cluster.mu/) 8SHOWROOM(https://note.com/marketing/n/ 12Kaguna Luna is a Japanese VTuber nc1db5ce8d937) 13Introduction and table of contents(https://note.com/shogo_vr/n/ 9 Mirrativ(https://www.mirrativ.com/) naa3fc070228e)

499 69.2% gathered based on common hobbies and preferences; it also 23rd, 2018. Not wanting to be assigned to a narrow booth, taking into appeared that they would be involved in Discord. 34.9% of users account safety regulations, long time period to complete art pieces, had experienced romance while 47.7% expressed that they had no high price and work explanations that come with the real world, interest in romance. Yotsumi Flame created this exhibition where art could be created Regarding the economic zone of VRChat, Virtual Market had a quickly, at low cost with venue designed according to personal significant contribution to the emergence of avatar economy. Before preferences. Twitter was used to advertise the exhibition as well the event’s existence, avatar sales were at 19.5%, where’s after the as to interact with visitors. The artist created a VRChat world first event they went up to 47.7%. Virtual Market contributed to a specifically for this exhibition in a traditional, realistic “white cube” culture of purchasing and selling avatars and accessories on VRChat. format for art objects to stand out. This is a unique feature to virtual 60% of users buy the avatars and customize them to their own taste art exhibitions as artists can create artworks and exhibition venues. and 95.6% of VRChat users had experience in purchasing or creating At the entrance of the museum, recommended avatar height as well their own avatar. Most purchases are made on Booth website. Booth as specifications to museum etiquette were inscribed: users could is a Japanese website for selling various contents. HTC Vive was chat and take photos while freely moving around. The concept of the also the most commonly used HMD, followed by Oculus Quest and exhibition was to interrogate the boundaries between the real and the Valve Index. virtual worlds. Approximately 20 art pieces were displayed. Some incorporates shaders or particles; some would take on different visual 4.2 Virtual Market aesthetics depending on visitors interactions with them; VTuber Hosted by Tokyo-based VR company Hikky, Virtual Market14 is a channels with their number of subscribers and relationships between series of virtual trade and exhibition conventions held in VRChat that them were represented by particles; an art piece would demonstrate have begun in 2018. Virtual Market will hold its fourth exhibition the actual physical distance between participants; with some in was in 2020 (Figure1). The first version was created from the desire to possible to interact; some were viewed differently depending on provide a meeting space where people who make avatars and those visitors IP address and some were more modern art-like. Museum who use them would connect; as well as from the “Doujinshi” model shops are custom to real life venues, however in the case of this of self-published conventional “comic market”. Unlike the regular virtual one, all items exposed during the exhibition were sold for “comic market”, Virtual Market is not designed as a flat space with free. By scanning the QR code, visitors could download the artwork rows of booths. The goal of the event is to enrich and push the from Booth. boundaries of VR. As it is held in VRChat, it is a hybrid event with participants accessing it from desktop and VR. The intricate 4.4 VR Architecture Award (VRAA) maze-like design is a by-product of technological weight limitations. VR Architecture Award15 concept was to challenge creatives to think The event is accessible during several days and unlike a real-world about architecture as living spaces that can take any form in VR. exhibition, does not have opening hours. Visitors are not restricted by their time zones and can access Virtual Market at any time and freely browse through virtual goods, clothes, props, and try on virtual avatars and buy 3D CG content. Creators, such as individuals, VTubers or avatar related companies, can interact with the audience by presenting their creations in booths commonly decorated with posters, 3D objects or interactive contents. In-game purchasing is not available; however, all booths have a QR code and links to Booth storefront where the items can be acquired. Not all creators can exhibit their work as the number of available booths is limited. Virtual Market is a multi-site event where booths are organized in different venues across different worlds with individual themes.

Figure 2: VRAA YouTube Archive by Ocutan-bot.

In this contest, architecture does not refer to the strict terms of“buildings” and “houses”, but rather to architecture as a mean for creating liveable spaces as a design philosophy. As more and more people spend their time in VRChat, the spaces (worlds) they oc- cupy take on the role of liveable environments like in reality. Those worlds are 3-dimensional constructions that are full of possibilities that go beyond reality, renewing the notion and visible aspects of what we consider liveable spaces. The committee chair had three individuals who judged submitted architectural world proposals. Figure 1: Key visual of Virtual Market 2nd edition. Criteria were decided upon by the committee: originality, richness of spatial expression, how the spaces could be used and the actual results, proposal’s impact on real world and society. 107 registra- tions were made with a total of 61 works, but only 10 proposals 4.3 Media art exhibition made it to the final round16. Judges immersed themselves in the Yotsumi Flame, the individual behind the first media art exhibition in worlds, experiencing and walking around them before making the VRChat, held “1% of virtuality” from September 22nd to September 15VR Architecture Award(https://xrarchi.org/vraa1/) 14Virtual Market(www.v-market.work/v4/) 16List of submitted boards(http://xrarchi.org/vraa1/worklist/)

500 final judgement17. The VRChat event was broadcast-ed on YouTube and archived versions are still available (Figure2). This event chal- lenged the conventional forms of architecture, living spaces design, content judgement and viewer accessibility. Living space design is no longer limited to high budget design studios and specific loca- tions, they are now immersive environments that can be accessed by anybody. VRAA used VRChat to hold and support a multi-site event (different worlds) which addressed the questions of design, process and production of presentations in social VR as it asked the participants to rethink the design of social, liveable spaces that were presented as worlds.

5CASE STUDY OF CONFERENCES HELD ON EACH PLAT- FORM 5.1 Case 1: Hybrid workshop “VRSionUp” VRSionUp is a set of VR research workshops curated by GREE VR Figure 3: VRSionUp#5 YouTube Archive by VR Studio Lab GREE. Studio Lab on the theme of “Discovering innovation through VR” by GREE VR Studio Lab. International and Japanese creators are in- vited to present on specific technologies (How), on concepts behind live entertainment by virtual beings” [21] was part of the Real- innovations (Why), and on the meanings that they hold (What). VR- Time Live session. was part of the Real-Time Live session. On SionUp workshops are a place where academia, entertainment and SIGGRAPH official website the event is described as “a showcase technology meet in a pursuit for innovation. Six hybrid workshops of a next generation of live entertainment using global bidirectional have been organized in the first half of 2019. Following are the 18 remote network and haptic displays for VTubers. Performers will individual themes: VRSionUp1 (January 11, 2019) : High school play 3D real-time characters. The remote players harmonize the VR research at international conference, VRChat in science research, 19 stage by applauding and cheering on the performer on site. Emotion Voice changer potentials. VRSionUp2 (March 1, 2019) :LavalVir- will be visualized for various audiences which includes audio and or tual 2019, Virtual Market 2 preparatory meeting. VRSionUp3 (April visual disabilities” [21]. 12, 2019)20: Laval Virtual 2019 official report, VTuber program 21 This multi-site event united together performers who were in Bris- technological features. VRSionUp4 (May 17, 2019) : VR entertain- bane and Tokyo on the same Virtual Cast stage. They interacted via ment x Edu. VRSionUp5 (June 21, 2019)22: Kawaii move research. 23 Virtual Cast with the live audience of SIGGRAPH. The conference VRSionUp6 (July 16, 2019) : Advanced voice changer research. had a script that began with a presentation of the project in the real Workshops format is often a mix of keynote speeches by specialists world and a transformation into a virtual avatar. Next, on a giant like pantomimer, singer, performer and top runner researchers, lec- screen, the Virtual Cast room was broadcast-ed in real time, with tures, reports, technological demonstrations and “Lightning Talk”, four participants: Lea, MIlky Queen, Killin and Xi (Figure4). Kilin short student interventions. Participants present their latest work and Xi are AI idols who are building bridges between Japan and via Google Slides in real-time, videos, physical objects or streamed China through live animation. The participants interacted in English, content. Japanese and Chinese and by using a translation system, could easily Based on each workshop’s theme, various guests are invited to switch between languages. The four participants explained Virtual present their research and work in a hybrid workshop that is held Cast and the technology behind it. As it took place simultaneously in both physical locations and VR. The events take place at GREE in the physical space of SIGGRAPH and the VR environment, “Hap- headquarters in Roppongi, Tokyo. Being a multi-site event, guests beat” [27] was used to create interactions between physical and are invited to present their work in physical form at the event or they virtual venues that went beyond a simple image transmission. can also broadcast themselves and their research material from a Performers proceeded to explain “Hapbeat”: a haptic device private VRChat room. All workshops were free and were broadcast- that enhances emotions for real time entertainment. “Hapbeat” is ed on YouTube Live where viewers could ask questions in real time a necklace-type wearable device that transmits sound vibrations via the chat. For those who could not participate at the physical onto the body, providing an immersive sound experience. With location, it was possible to engage with the conference through a it, VTubers can not only see but also feel the audience’s emotions. designated VRChat room where VRSionUp was streamed (Figure3). While hugging or shaking hands, “Hapbeat” transmits in real-time The audience could participate in the Q & A from Roppongi venue sensations to the performers. The device could also analyse the or from the VRChat room, interacting between physical and virtual sound emitted from the audience, such as clapping and excitements spaces. and transmit the sensations to the performers. The script ended with a game as to showcase an entertaining way to participate in common 5.2 Case 2: Virtual Cast: SIGGRAPH Asia 2019 Real activities in a long-distance interaction. By showcasing what virtual Time Live! beings can do together in a VR society, this hyper virtually created In SIGGRAPH ASIA 2019 Brisbane, “Real-Time Live!” event a participatory experience with interactions between audience and (SA19RTL) was organised by ACM SIGGRAPH to showcase best performers in Virtual Cast. real-time graphics interactivity. “Global bidirectional remote haptic 5.3 Case 3: Virtual Reality Specialist Certification Test- 17VRAA video archive(http://xrarchi.org/vraa1/video/) ing unofficial Study Session (in Cluster) 18VRSionUp1(https://gree.connpass.com/event/112901/) On November 24, 2019 an event “Let’s Become VR specialist”24 19VRSionUp2(https://gree.connpass.com/event/117456/) was organized as an unofficial study and a presentation session to pre- 20VRSionUp3(https://gree.connpass.com/event/122745/) pare for the “VR technician/specialist certification test” conducted 21VRSionUp4(https://gree.connpass.com/event/128401/) 22 VRSionUp5(https://gree.connpass.com/event/131231/) 24Let’s Become VR specialist(https://cluster.mu/events/ 23 VRSionUp6(https://gree.connpass.com/event/134082/) 7ec139eb-97e6-429f-89ea-8638ef599d5c)

501 society. The organizer believed that sharing and creating values from the academia’s perspective, in terms of knowledge, will be a universal value in the physical world. The event was free and operated by volunteers: they created a place for communication that resolved the gap between perceptions of academia and audience. The first edition had an opening cere- mony, poster session core time (Figure6), keynote speech (Figure7), closing ceremony (Figure8), and a social gathering as part of the official program. The keynote speakers focused on the history of VR technology, its future development, and new ways of using it. “Phio Alchemist”, a keynote speaker and virtual event producer of Hikky, gave a talk on their strategy. It contained new user generated content driven market that caters towards an audience interested in VR as their life environment. Poster session had 12 accepted speakers and was archived by Ocutan-bot (@oculustan) on YouTube. Figure 4: SA19RTL - Global Bidirectional Remote Haptic Live En- All events were distributed by YouTube Live, and enthusiastically tertainment by Virtual Beings YouTube Archive by VR Studio Lab discussed on VRChat, Cluster, and Twitter. GREE. by the Virtual Reality Society of Japan. The study session used Cluster, speakers were volunteers and the number of participants was not limited. Even though the event was held in VR, the hybrid format made it accessible from PC and Mac. The Study Session was broadcast-ed on YouTube Live and viewers comments were shown in the Cluster venue. An archive was saved on YouTube and was made available even after the event has ended. The duration of the presentations was of 55 min, with speakers’ PDF slides (movies and gifts were not allowed) projected on the screen in the Cluster event. As it was broadcast-ed on YouTube live, discussions between view- ers and presenters were encouraged after presentations (Figure5). Study Session is a unique approach to academic conferences as it utilized the VR environment for students to practice their presenta- tions before an exam and have direct feedback from people all over Figure 6: Poster session of “Virtual Gakkai” in VRChat. the world.

Figure 5: Virtual Reality Specialist Certification Testing unofficial Study Session YouTube archive by VRM Study. Figure 7: Keynote session of “Virtual Gakkai” by Prof. Masahiko INAMI in Cluster. 5.4 Case 4: Virtual Gakkai The first edition of “Virtual Gakkai” [7]25was organized on Decem- 6CONCLUSION ber 14th, 2019 by Takayuki Kameoka, a master student in UEC (Co-author of this article). He has contributed to IVRC 2015 by an As it has been introduced in this research paper, “Avatar Driven innovative project “Urea labyrinth” — a sensible urinate VR sim- VR society” is developing well in Japanese VR environment. It ulator26. The purpose of Virtual Gakkai was to hold a conference is based on consumer VR, such as VR3.0, well developed plat- on Virtual Reality Social Networking Service (VRSNS) such as forms, imaginary personas (avatars) and engagement-based identity VRChat and Cluster in the form of an actual conference. The main interactions on social media platforms like Twitter. However, vir- purpose was to explore forms of value creation in the cyber-world, tual identities are not only limited to a realistic appearance like Lil 27 which would then serve as a basis for the upcoming avatar and cyber Miquela . Anime-tic characters are also used in China, Korea, Taiwan, Hong Kong, Thailand and some eastern Asian countries. 25Gakkai: academic conference in Japanese Hitachi Global Life Solutions released their VTuber Shirokaden 26 Urealabyrinth(https://urealabyrinth.wixsite.com/ 27 incontinence?lang=en) Lil Miquela(https://www.instagram.com/lilmiquela/)

502 [9] A. Leavitt, T. Knight, and A. Yoshiba. Producing hatsune miku: Con- certs, commercialization, and the politics of peer production. In P. W. Galbraith and J. G. Karlin, eds., Media Convergence in Japan, pp. 200–232. Kinema Club, 2016. [10] B. Lufkin. The virtual vloggers taking over youtube. https://www.bbc.com/worklife/article/20181002-the-virtual-vloggers- taking-over-youtube, Oct 2018. [11] J. McVeigh-Schultz, E. Marquez´ Segura, N. Merrill, and K. Isbister. What’s it mean to ”be social” in vr?: Mapping the social vr design ecology. In Provocations and Work-in-Progress, pp. 289–294. DIS’18 Companion, Hong Kong, 2018. doi: doi:10.1145/3197391.3205451 [12] H. Moore. Avatars and robots: the imaginary present and the socialities of the inorganic. Cambridge Anthropology, 30(1):48–63, 2012. doi: doi:10.3167/ca.2012.300106 [13] J. Morie. Performing in virtual spaces: Embodiment and being in virtual environments. International Journal of Performance Arts and Digital Media, 3:123–138, 2007. doi: doi:10.1386/padm.3.2-3.123 1 [14] P. Nagy and B. Koles. The digital transformation of human identity: Towards a conceptual model of virtual identity in virtual worlds. Con- vergence, 30(3):276–292, 2014. doi: doi:10.1177/1354856514531532 [15] K. Pavitt. Innovation processes. In J. Fagerberg and D. C. Mowery, eds., Figure 8: Ending of keynote session of “Virtual Gakkai” in Cluster, The Oxford Handbook of Innovation, pp. 86–113. Oxford University there were various avatars in the audience. Press, Oxford, 2009. [16] E. A. Schegloff. On talk and its institutional occasions. In P. D. . J. 28 Heritage, ed., Talk at Work: Interaction in Institutional Settings, pp. Hakushaku (Earl of white appliances) as a brand promotional char- 101–134. Cambridge University Press, Cambridge, 1992. acter. Many companies use VTubers for various communication [17] J. A. Schumpeter. The theory of economic development: an inquiry purposes. VRChat is also becoming an immense creation market into profits, capital, credit, interest and the business cycle. Harvard thanks to semi-professional content creators. Economic Studies, 46, 1934. doi: doi:10.1177/1354856514531532 Hybrid workshops create opportunities to meet creators face to [18] A. Shirai. “reality: Be yourself” vtuber brings a new ecosystem in face and most of online/offline events are operated by student or vr4.0 era. https://fmx.de/program2019/event/15042, May 2019. voluntary staff. Such environment enables synergy and rapid updates [19] A. Shirai. Reality: Broadcast your virtual beings from everywhere. in a short period of time. The first edition of “Virtual Conference” In ACM SIGGRAPH 2019 Appy Hour, SIGGRAPH ’19. Association was organized in the end of 2019. New issues might arise if “Virtual for Computing Machinery, New York, NY, USA, 2019. doi: 10.1145/ Conference” publishes a scientific journal; multiple authorship and 3305365.3329727 certification of identity (participants used avatars and not their real [20] A. Shirai, J. Nishida, D. Sato, K. Sato, H. Takatori, and S. Tachi. In- identities) will have to be approached differently and thought of in novation in the age of virtual reality through organizing international a new format. In another possibility, we, the authors of this article student competition. In SIGGRAPH Asia 2015 Symposium on Educa- are written by unknown researchers who are organized around the tion, SA ’15. Association for Computing Machinery, New York, NY, same vision. An author had purchased new VR headsets to present USA, 2015. doi: 10.1145/2818498.2818505 [21] A. Shirai, Y. Yamazaki, K. Koike, and Y. Soejima. Global in an avatar conference. As evidence, avatar driven VR will bring bidirectional remote haptic live entertainment by virtual be- society, with people not ready for VR VR3.0, to the next generation ings. https://sa2019.siggraph.org/attend/real-time-live/session/111, of virtual reality. It may be seen as a singularity from the possible Nov 2019. future. [22] K. Sledzik.´ Schumpeter’s view on innovation and entrepreneurship. In S. Hittmar, ed., Management Trends in Theory and Practice, pp. 85–95. REFERENCES University of Zilina Institute of Management, University of Zilina, [1] M. Castells. Communication, power and counter-power in the network 2013. society. International Journal of Communication, 1(1):238–266, 2007. [23] S. Tachi. From 3d to vr and further to telexistence. In 2013 23rd doi: doi:1932-8036/20070238 International Conference on Artificial Reality and Telexistence (ICAT). [2] B. Coleman. Hello Avatar: Rise of the Networked Generation. The IEEE, Tokyo, Japan, 2013. doi: 10.1109/ICAT.2013.6728898 MIT Press, Massachusetts, London, 2002. [24] T. Taylor. Living digitally: Embodiment in virtual worlds. In [3] I. collegiate Virtual Reality Contest (IVRC). Ivrc archives. R. Schroeder, ed., The Social Life of Avatars, pp. 40–62. Springer, http://ivrc.net/archive/, 1993-2019. London, 2002. [4] M. Dawson. Embedded and situated cognition. In L. Shapiro, ed., [25] L. Van Langenhove. Introducing positioning theory. In L. Van Langen- The Routledge Handbok of Embodied Cognition, pp. 59–68. Routledge, hove, Harre´ Van Langenhove, ed., Positioning Theory: Moral Contexts New York, London, 2014. of Intentional Action, pp. 14–31. Blackwell Publishers, New Jersey, [5] J. Fagerberg. Schumpeter and the revival of evolutionary economics: 1999. An appraisal of the literature. Journal of Evolutionary Economics, [26] A. M. von der Putten, N. C. Kramer,¨ and J. Gratch. How our personality 13:125–159, 2003. doi: doi:10.1007/s00191-003-0144-1 shapes our interactions with virtual characters — implications for [6] J. Guga. Virtual idol hatsune miku - new auratic experience of the research and development. In J. Allbeck, N. Badler, T. Bickmore, performer as a collaborative platform. In Arts and Technology, pp. C. Pelachaud, and A. Safonova, eds., Intelligent Virtual Agents, pp. 36–44. Springer, 2014. 208—-221. Springer, Berlin, Heidelberg, 2010. [7] T. Kameoka. Virtual gakkai homepage. [27] Y. Yamazaki, S. Hasegawa, H. Mitake, and A. Shirai. Neck strap https://sites.google.com/view/virtualconference-2019/home, Dec 2019. haptics: An algorithm for non-visible vr information using haptic [8] H. Kennedy. Beyond anonymity, or, future directions for internet perception on the neck. In ACM SIGGRAPH 2019 Posters, SIGGRAPH identity research. New Media & Society - NEW MEDIA SOC, 8(1):859– ’19. Association for Computing Machinery, New York, NY, USA, 2019. 876, 2006. doi: doi:10.1177/1461444806069641 doi: 10.1145/3306214.3338562

28Shirokade(https://www.youtube.com/channel/ UCW5nevekF7CWtKgpNMz9ACA)

503