A Survey of Surface Computing

Research Manuscript Title

A SURVEY OF SURFACE COMPUTING

MS.K.RANJITHA., Mrs.M.JOTHILAKSHMI.,M.Sc.,M.Phil

Research Scholar, Assistant Professor, Department of Computer Science & Application,

Vivekanandha College of Arts & Sciences for Women, Elayampalayam, Tiruchengode.

www.istpublications.com

Ms.k.ranjitha., Mrs.M.jothilakshmi, ―A SURVEY OF SURFACE COMPUTING‖, International Journal of Future Innovative Science and Engineering Research (IJFISER) ISSN (Online): 2454- 1966, Volume-2, Issue-1, March - 2016, Page | 21

A SURVEY OF SURFACE COMPUTING

MS.K.RANJITHA., Mrs.M.JOTHILAKSHMI.,M.Sc.,M.Phil

Research Scholar, Assistant Professor, Department of Computer Science & Application, Vivekanandha College of Arts & Sciences for Women, Elayampalayam, Tiruchengode.

ABSTRACT

The name Surface comes from "surface computing," and Microsoft envisions the coffee-table machine as the first of many such devices. Surface computing uses a blend of wireless protocols, special machine-readable tags and shape recognition to seamlessly merge the real and the virtual world — an idea the Milan team refers to as "blended reality." The table can be built with a variety of wireless transceivers, including Bluetooth, Wi-Fi and (eventually) radio frequency identification (RFID) and is designed to sync instantly with any device that touches its surface. It supports multiple touch points – Microsoft says "dozens and dozens" -- as well as multiple users simultaneously, so more than one person could be using it at once, or one person could be doing multiple tasks. The term "surface" describes how it's used. There is no keyboard or mouse. All interactions with the computer are done via touching the surface of the computer's screen with hands or brushes, or via wireless interaction with devices such as smart phones, digital cameras or Microsoft's Zune music player. Because of the cameras, the device can also recognize physical objects; for instance credit cards or hotel "loyalty" cards.

Keywords: Interactive surfaces and tabletops, multi-touch, physics-simulation, field-study, home.

1.Introduction

A surface computer is a computer that interacts with the user through the surface of an ordinary object, rather than through a monitor and keyboard. Surface computing is the term for the use of a specialized computer GUI in which traditional GUI elements are replaced by intuitive, everyday objects. Instead of a keyboard and mouse, the user interacts directly with a touch-sensitive screen. It has been said that this more closely replicates the familiar hands on experience of everyday object manipulation. With Surface, Microsoft has established a new branch of computer technology known as surface computing. The goal of surface computing is to recognize touch and objects on the screen's surface and to interact with those objects seamlessly. If you're using a surface computer, you shouldn't need a mouse, keyboard or even a USB port connected to the device. In this paper we pre- sent one of the first deployments of a multi-touch tabletop device in a domestic setting. Our overall goal is to investigate the potential for surface computing in the home. Specifically, in this paper we seek to observe what people‗s interactions, perceptions and experiences are of such novel computing technologies and interfaces, as a means to fur-their inform the design space.

Over the past couple of years, a new class of interactive device has begun to emerge, what can best be described as ―surface computing‖. Two examples are illustrated in this report. They are-

• Surface Table top

• Perceptive Pixel

The Surface table top typically incorporates a rear-projection display coupled with an optical system to capture touch points by detecting shadows from below. Different approaches to doing the detection have been used, but most employ some form of IR illumination coupled with IR cameras. With today‘s camera and signal processing capability, reliable responsive and accurate multi-touch capabilities can be achieved.

2. Literature Survey

For years engineers and computer technicians have looked for a better way for people to communicate with their computers. Keyboards while feeling natural to many of us has advanced very little beyond the typewriters which have been around for well over a hundred years and though the mouse is a step above that it still takes practice for someone who has never used one to become used to the idea of moving the mouse with it and after years of using a computer many older people still have trouble with the concepts of double clicking, right clicking, dragging, dropping and other techniques that can seem simple to more advanced computer users. The most recent solution, and one that seems likely to stick is that of surface computing. Surface computing at its most basic is an attempt to make the use of a computer better match the way we interact with other things in our environments as well as better interacting with those things and allowing for far less time thinking about how we interact with our computers so more energy can be put into how we use them.

The most common and popular type of surface computing is that of touch screen monitors of the type that can be found on many modern phones. These are also common in many businesses where untrained workers are expected to use a computer. Until recently though these touch screen monitors were really little more than a replacement of the mouse. You could still only point at one thing at a time, and it wasn't even as good as a mouse because you can't right click or highlight things without using a keyboard. The key difference between this and other similar devices is the multi touch system. This allows users to use both hands to manipulate things such as photos, as well as the use of hand gestures and even physical gestures. Other features on the Microsoft surface allows for wireless communicates between devices so things like phones, cameras and laptops can sync with the table and move data between them effortlessly. Surface computing is a powerful movement. In fact, it‘s as significant as the move from DOS [Disk Operating System] to GUI [Graphic User Interface]. Our research shows that many people are intimidated and isolated by today‘s technology. Many features available in mobile phones, PCs and other electronic devices like digital cameras aren‘t even used because the technology is intimidating. Surface computing breaks down those traditional barriers to technology so

that people can interact with all kinds of digital content in a more intuitive, engaging and efficient manner. It‘s about technology adapting to the user, rather than the user adapting to the technology. Bringing this kind of natural user interface innovation to the computing space is what Surface Computing is all about.

2.1How It All Started

In 2001, Stevie Bathiche of Microsoft Hardware and Andy Wilson of Microsoft Research began working together on various projects that took advantage of their complementary expertise in the areas of hardware and software. In one of their regular brainstorm sessions, they started talking about an idea for an interactive table that could understand the manipulation of physical pieces. Although there were related efforts happening in academia, Bathiche and Wilson saw the need for a product where the interaction was richer and more intuitive, and at the same time practical for everyone to use. This conversation was the beginning of an idea that would later result in the development of Surface, and over the course of the following year, various people at Microsoft involved in developing new product concepts, including the gaming-specific PlayTable, continued to think through the possibilities and feasibility of the project.

Fig 2.1 Surface Computing In 2003(T1 Prototype)

2.2 Hardware Design:

By late 2004, the software development platform of Surface was well established and attention turned to the form factor. A number of different experimental prototypes were built including ―the tub‖ model, which was encased in a rounded plastic shell, a desk-height model with a square top and cloth-covered sides, and even a bar-height model that could be used while standing.

Fig 2.2 Consensus on a group Artifact

2.3 From Prototype to Product:

The next phase of the development of Surface focused on continuing the journey from concept to product. Although much of what would later ship as Surface was determined, there was significant work to be done to develop a market-ready product that could be scaled to mass production. In early 2006, Pete Thompson joined the group as general manager, tasked with driving end-to-end business and growing development and marketing. Under his leadership, the group has grown to more than 100 employees. Today Surface has become the market-ready product once only envisioned by the group, a 30-inch display in a table like form factor that‘s easy for individuals or small groups to use collaboratively. The sleek, translucent surface lets people engage with Surface using touch, natural hand gestures and physical objects placed on the surface.

2.3THE UNDERLYING TECHNOLOGY

The Core features of Surface Computer consists of:

2.3.1 Multi-touch user interaction- The horizontal form factor makes it easy for several people to gather around surface computers together, providing a collaborative, face‐to‐face computing

2.3.2 Tangible user interface-

Surface aims to provide physical form of digital information. i) One can draw on surface with any physical paint brush.

ii) placed on the Surface, squares of glass can play videos (puzzle).

Hand gestures are preferred to physical instruments.

2.3.3 Multi-user Interface-

Users can place physical objects on the surface to trigger different types of digital responses. They are identified by their shapes or embedded ID tags. Surface acts as a mediator between devices(e.g. data exchange between digital camera and Mobile phone).

2.3.4 Object Recognition-

Surface also features the ability to recognize physical objects that have identification tags similar to bar codes. This means that when a customer simply sets a wine glass on the surface of a table, a restaurant could provide them with information about the wine they‘re ordering, pictures of the vineyard it came from and suggested food pairings tailored to that evening‘s menu.

Fig 2.3 Total Internal Reflection Technology

2.4 TECHNOLOGY BEHIND IT

The main idea behind the surface technology is to let people interact with their digital content in a much more new and innovative way which is quite different from the traditional ways. The concept is to give digital content a new dimension where it is not restricted to your mobile phones or television sets rather has the ability to interact with you physically. Microsoft Surface uses cameras to sense objects, hand gestures and touch.

• Microsoft Surface uses a rear projection system which displays an image onto the underside of a thin diffuser.

• Objects such as fingers are visible through the diffuser by series of infrared– sensitive cameras, positioned underneath the display.

• An image processing system processes the camera images to detect fingers, custom tags and other objects such as paint brushes when touching the display.

One of the key components of surface computing is a "multitouch" screen. It is an idea that has been floating around the research community since the 1980s and is swiftly becoming a hip new product interface — Apple's new iPhone has multitouch scrolling and picture manipulation. Multitouch devices accept input from multiple fingers and multiple users simultaneously, allowing for complex gestures, including grabbing, stretching, swivelling and sliding virtual objects across the table.

Fig 2.4 Our interactive surface, a bespoke tabletop, with an embedded multi-touch display and overhead camera for imaging physical objects on the surface (left), with a custom physics-enabled photo application (right).

2.5 MICROSOFT SURFACE OVERVIEW

Microsoft Surface turns an ordinary tabletop into a vibrant, interactive computing experience. The product provides effortless interaction with digital content through natural gestures, touch and physical objects. In Essence, it‘s a surface that comes to life for exploring, learning, sharing, creating, buying and much more. Currently available in select in restaurants, hotels, retail establishments and public entertainment venues, this experience will transform the way people shop, dine, entertain and live. The intuitive user interface works without a traditional mouse or keyboard, allowing people to interact with content and information by using their hands and natural movements. Users are able to access information either on their own or collaboratively with their friends and families, unlike any experience available today.

2.5.1 The Hardware:

Essentially, Microsoft Surface is a computer embedded in a medium-sized table, with a large, flat display on top that is touch-sensitive. The software reacts to the touch of any object, including human fingers, and can track the presence and movement of many different objects at the same time. In addition to sensing touch, the Microsoft Surface unit can detect objects that are labelled with small "domino" stickers, and in the future, it will identify devices via radio-frequency identification (RFID) tags. The demonstration unit I used was housed in an attractive glass table about three feet high, with

a solid base that hides a fairly standard computer equipped with an Intel Core 2 Duo processor, an AMI BIOS, 2 GB of RAM, and Windows Vista.

(1) Screen: A diffuser turns the Surface's acrylic tabletop into a large horizontal "multitouch" screen, capable of processing multiple inputs from multiple users. The Surface can also recognize objects by their shapes or by reading coded "domino" tags.

(2) Infrared: Surface's "machine vision" operates in the near-infrared spectrum, using an 850- nanometer-wavelength LED light source aimed at the screen. When objects touch the tabletop, the light reflects back and is picked up by multiple infrared cameras with a net resolution of 1280 x 960.

(3) CPU: Surface uses many of the same components found in everyday desktop computers — a Core 2 Duo processor, 2GB of RAM and a 256MB graphics card. Wireless communication with devices on the surface is handled using WiFi and Bluetooth antennas (future versions may incorporate RFID or Near Field Communications). The underlying operating system is a modified version of Microsoft Vista.

(4) Projector: Microsoft's Surface uses the same DLP light engine found in many rearprojection HDTVs. The footprint of the visible light screen, at 1024 x 768 pixels, is actually smaller than the invisible overlapping infrared projection to allow for better recognition at the edges of the screen.

3.FUTURE ENHANCEMENT:

Multi-touch display. The Microsoft Surface display is capable of multi-touch interaction, recognizing dozens and dozens of touches simultaneously, including fingers, hands, gestures and objects.

Perceptive Pixel‘s touch screens work via frustrated total internal reflection Technology. The acrylic surface has infrared LEDs on the edges. When undisturbed, the light passes along predictable paths, a process known as total internal reflection. When one or more fingers touch the surface, the light diffuses at the contact points, changing the internal-reflection pathways. A camera below the surface captures the diffusion and sends the information to image-processing software, which translates it into a command. Multitouch technology has been around since early research at the University of

Toronto in 1982.

Vendors are beginning to release commercial multitouch systems. For example, Mitsubishi Electric Research Laboratories‘ Diamond Touch table, which includes a developer‘s kit, can be used for small-group collaboration.

Horizontal orientation:

The 30-inch display in a table-sized form factor allows users to share, explore and create experiences together, enabling a truly collaborative computing experience.

Dimensions.

Microsoft Surface is 22 inches high, 21 inches deep and 42 inches wide.

Materials.

The Microsoft Surface tabletop is acrylic, and its interior frame is powder-coated steel.

Perceptive Pixel

Computer scientists see technologies such as surface computing and multitouch as the key to a new era of ubiquitous computing, where processing power is embedded in almost every object and everything is interactive. Last year, New York University professor Jeff Han launched a company called Perceptive Pixel, which builds six-figureplus custom multitouch drafting tables and enormous interactive wall displays for large corporations and military situation rooms. "I firmly believe that in the near future, we will have wallpaper displays in every hallway, in every desk.

Fig 3.0 Demo of Perceptive Pixel’s Multi-touch

The display‘s surface is a six-millimeter-thick piece of clear acrylic, with infrared LEDs on the edges. Left undisturbed, the light passes along predictable paths within the acrylic, a process diffuses at the contact point, causing the acrylic‘s internal reflection pathways to change. A camera below the surface captures the diffusion and sends the information to image-processing software, which can read multiple touches simultaneously and translate them into a command.

4. APPLICATIONS

4.1Video Puzzle

Video Puzzle showcases the power of the little identification tags mentioned above. The tags consist of a pattern of variously-sized dots; Keam mentioned that the dots currently represent an 8-bit code (256 permutations) but that 128-bit tags were in the works. Not only can the tags transmit numerical information, but the geometrical arrangement of the dots means that Surface can also tell, to a high degree of accuracy, how much the tag (and therefore the object) has rotated. In Video Puzzle, these

virtually invisible tags are placed upon small squares of glass. When the pieces of glass are put on the table, the screen starts playing video clips underneath each one. Because the video moves whenever you move the squares, it creates the illusion that the glass itself is displaying the video, which looks very futuristic.

Fig 4.1 Video Puzzle

4.2 Paint

Paint programs have been a natural demonstration application for new platforms ever since MacPaint graced the first Macintosh back in 1984. Surface‘s paint program is even lighter on features than MacPaint was, but the natural user interface makes up for this deficiency.

Fig 4.2 Paint Application

4.3 Music

The Music application works like a virtual jukebox, displaying music arranged by album and allowing the user to flip over albums, select songs, and drag them to the "Now Playing" section. The album browser works a bit like Apple‘s Cover Flow, although many albums are visible at once without scrolling.

Fig 4.3 Music Application

4.4 Minor Reports of Surface Computers

The Rosie Surface Computer is an Apple based surface computer. It was created by Savant home A/V computer and uses a 40 inch coffee table running apple technology to create integration between computing and everyday tasks. The Savant surface is an evolutionary technology that can download and share photos from digital cameras, and also play music, movies, TV shows right onto its surface and accomplish complete home control all from within one elegant interface. Savant plans to offer the ROSIE Surface in a number of different furniture styles ranging from contemporary to traditional. The hardware specs haven‘t been released yet. However some features that have been verified are:

 Interaction with iTunes multimedia content, digital cameras, IP network  cameras, business card readers, and many more digital devices on the horizon.  Touchscreens readily created & customized via RacePoint Blueprint design tool.  Customizable Hot Launch Buttons.  Custom tailored to suit any décor with the ROSIE Surface Kit.

Fig 4.4 APPLE Rosie surface computer

Conclusion:

Although surface computing is a new experience for consumers, over time Microsoft believes there will be a whole range of surface computing devices and the technology will become pervasive in people‘s lives in a variety of environments. As form factors continue to evolve, surface computing will be in any number of environments— schools, businesses, homes — and in any number of form factors — part of the countertop, the wall or the refrigerator. Some people will look at Surface and claim that it does nothing that hasn't been tried before: computers with touch screens have been around for years and have already found niches in ATMs, ticket ordering machines, and restaurant point-of-sale devices.

REFERENCES

[1] Agarawala, A. and Balakrishnan, R. 2006. Keepin' it real: pushing the desktop metaphor with physics, piles and the pen. In Proc. CHI '06 (April 22 - 27, 2006). [2] Apted, T., Kay, J., and Quigley, A. 2006. Tabletop sharing of digital photographs for the elderly. In Proceedings CHI '06). [3] Buxton, B., Multi-touch systems that I know and love, http://www.billbuxton.com/ multitouchOverview.html. [4] Guiard. Y., Asymmetric division of labor in human skilled bimanual action: The kinetic chain as a model. The Journal of Motor Behavior, 19(4):486–517, 1987. [5] Han, J. Y. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceed-ings of UIST '05). [6] Hancock, M., et al. 2010. Supporting sandtray therapy on an interactive tabletop. In Proc. CHI '10. [7] Hornecker, E., ―I don‗t understand it either, but it is cool‖ – Visitor Interactions with a Multi- Touch Table in a Museum. Proc. of IEEE Tabletop 2008. [8] Jacob, R.J. et al. Reality-based interaction: a frame-work for post-WIMP interfaces. In CHI '08. ACM, New York, NY, 201-210. [9] Kirk, D. et al. 2010. Opening up the family archive. In Proc. Of CSCW '10. ACM, New York, NY, 261-270. [10] Mazalek, A., et al. The TViews Table in the Home, In Proc. IEEE TABLETOP'07.

[11] Marshall, P, et al (2009) Fighting for control: children's embodied interactions when using physical and digital representations. In Proc. CHI '09, 2149-2152. [12] Morris, M.R., Brush, A.J., Meyers, B., A Field Study of Knowledge Workers‗ Use of Interactive Horizontal Displays. In Proc. IEEE Tabletop 2008. [13] Peltonen, P. et al. It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre. In CHI '08 (Florence, Italy, April 05 - 10, 2008). [14] Piper, AM & Hollan, J. D. (2009) Tabletop displays for small group study: affordances of paper and digital materials. In Proc. of CHI '09. ACM, 1227-1236. [15] Scott, S. D., Grant, K. D., and Mandryk, R. L. 2003. System guidelines for co-located, collaborative work on a tabletop display. In Proceedings ECSCW '03). [16] Scott, S. D., et al. Territoriality in collaborative tab-letop workspaces. In Proc. of ACM CSCW '04. [17] Shen, C., Lesh, N., and Vernier, F. 2003. Personal digital historian: story sharing around the table. Inter-actions 10, 2 (Mar. 2003), 15-22. [18] Shen, C., et al. 2006. Informing the Design of Direct-Touch Tabletops. IEEE Computer. Graph. Appl. 26, 5 (Sep. 2006) [19] Nielsen, J. (1994). Usability engineering. CA, USA: Morgan Kaufman. [20] Piper, A. M., O'Brien, E., Morris, M. R., & Winograd, T. (2006), SIDES: A cooperative table top computer game for social skills development. In Proceedings of the 6th International Conference on Computer Supported Cooperative Work, Alberta, Canada.