Ubiquitous Computing

Friedemann Mattern ETH Zurich Institute for Pervasive Computing

Eidgenössische Technische Hochschule Porquerolles May 2003 EETTH Zürich

Friedemann Mattern This lecture is not about algorithms or theoretical CS

Classical distributed algorithms (consistent state, termination detection, garbage collection,...) from a higher point of view

Ubiquitous Computing (pervasive computing, ambient intelligence,...): Almost unnoticed revolution that transforms the whole world into a large distributed system with important consequences to computer science... ...and to almost everyone living on earth

F. Ma. 2

Friedemann Mattern

image source: “Die Zeit”

Friedemann Mattern Size Number Computing: A Clear Trend

One computer (mainframe) for many people One computer (PC) for everyone

Many computers for everyone

F. Ma. 4

Friedemann Mattern

The Trend… What‘s Next?

Number smart dust? Many computers for everyone Size

F. Ma. 5

Friedemann Mattern The Qualitative Growth of the Internet

2003

Mobile Internet WWW Research Email network people to people to people machines Internet time line F. Ma. 6

Friedemann Mattern The Qualitative Growth of the Internet

Networked embedded systems machines talking 2003 to machines Ubiquitous Computing Mobile Internet WWW Research Email network people to people to machines to people machines machines Internet time line F. Ma. 7

Friedemann Mattern The Qualitative Growth of the Internet

Networked embedded systems machines talking 2003 to machines everyday objects Ubiquitous Computing Mobile Internet WWW Research Email network people to people to machines to people machines machines Internet time line F. Ma. 8

Friedemann Mattern Ubiquitous Computing

Information technology will be everywhere

Everyday objects will become smart embedded processors

...and they will all be interconnected wireless communication

F. Ma. 9 image source: “Die Zeit”

Friedemann Mattern

Outline

4 Technology Trends The Vision Making Things Smart Consequences

Friedemann Mattern, ETH Zurich F. Ma. 10

Friedemann Mattern

Outline

4 Technology Trends The Vision Making Things Smart Consequences

Friedemann Mattern, ETH Zurich F. Ma. 11

Friedemann Mattern

1. Moore‘s Law (1965)

F. Ma. 12

Friedemann Mattern Moore‘s Law Electronics, April 19, 1965

F. Ma. 13

Friedemann Mattern

Cramming more...

F. Ma. 14

Friedemann Mattern

Cramming more...

F. Ma. 15

Friedemann Mattern

Cramming more...

16 „...factor of two 15 14 per year…“ 13 12 11 10 „...by 1975, the 9

OF THE 8

number of components 2 7 6

per integrated circuit ... LOG 5 4 will be 65,000“ 3

NUMBER OF COMPONENTS 2

PER INTEGRATED FUNCTION INTEGRATED PER 1 0 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 YEAR

F. Ma. 16

Friedemann Mattern

Storage Density Trend

F. Ma. 18

Friedemann Mattern

Generalized Moore‘s Law

Most important technology parameters double every 1 – 3 years: Problems: computation cycles -increasingcost memory, magnetic disks -energy bandwidth

Consequence: scaling down

F. Ma. 19

Friedemann Mattern 2. Progress in Communication Technologies

Fiber optics: from Gbit/s to Tbit/s

Wireless mobile phone: GSM, UMTS wireless LAN (> 10 Mbit/s) Bluetooth

Body area networks

Nostalgia

F. Ma. 20

Friedemann Mattern Telecommunication and Information Everywhere – an Old Vision (1882)

Carl Stauber (1882): „Die Zukunft des Telefons“

F. Ma. 21

Friedemann Mattern

3. Better Sensors

Miniaturized cameras, microphones,... Fingerprint sensor Radio sensors without power supply Location sensors e.g., GPS ... POSITION N 047° 23’17’’ E008° 34’26’’ F. Ma. 22

Friedemann Mattern Example: Standalone Radio Sensors

No external power supply image source: Siemens energy from the actuation process piezoelectric and pyroelectric materials transform changes in pressure or temperature into energy RF signal is transmitted by an antenna (up to 20 m) F. Ma. 23

Friedemann Mattern

Piezoelectric Transponders

Transformer: electrical signal ↔ dipole antenna acoustic surface wave reflectors in both directions Surface wave: piezo- propagates on the crystal surface of a body electro acoustic on piezo crystals: transformer ~ 3500 m/s

Reflectors consist of 0.1 µm thinn aluminium stripes

F. Ma. 24

Friedemann Mattern

Piezoelectric Transponders

External energy pulse Transformed into a surface wave Each reflector sends parts of the wave back to the transformer Transformed into RF pulses and sent out

surface wave

reflections after a few µs F. Ma. 25

Friedemann Mattern

Piezoelectric Transponders

Surface wave is much slower than RF wave RF noise (e.g., reflections by the environment) vanishes after 1 µs RF response takes more than 2 µs

F. Ma. 26

Friedemann Mattern

Remote Identifications

Characteristic pulse sequence by specific alignment of the reflectors e.g., binary digits Æ up to 32 bits Æ identification of remote objects

F. Ma. 27

Friedemann Mattern

Remote Sensors

RF pulse

RF response

Second transformer at the end changes the impedance Can be controlled by a resistor that depends on some sensor value e.g., photo resistor, NTC/PTC resistor, hall sensor

F. Ma. 28

Friedemann Mattern

4. New Materials

Whole eras named after materials e.g., „Stone Age“ More recently: semiconductors, fibers information and communication technology first transistor, 1947

F. Ma. 30

Friedemann Mattern

4. New Materials

Whole eras named after materials e.g., „Stone Age“ More recently: semiconductors, fibers information and communication technology first transistor, 1947

Organic semiconductors Æ change the external appearance of computers „Plastic“ laser Æ opto electronics, flexible displays,… ... F. Ma. 31

Friedemann Mattern

Example: Flexible Substrates

F. Ma. 32

Friedemann Mattern

Smart Paper, Electronic Ink

Micro capsules Light state Dark state Top transparent electrode - + Negatively charged Positively charged black pigment chips white pigment chips

Clear fluid

Bottom electrode

+ -

F. Ma. 33

Friedemann Mattern

Smart Paper, Electronic Ink

Micro capsules Top transparent electrode

Negatively charged Positively charged black pigment chips white pigment chips

Clear fluid

Bottom electrode

+ -

0.2 mm

F. Ma. 34

Friedemann Mattern

Smart Paper, Electronic Ink

An electronically charged pencil rotates the “pixels”

F. Ma. 35

Friedemann Mattern All Trends Together Lead to a New Era

Progress in computing speed communication bandwidth material sciences Æ Pervasive Computing sensor techniques Æ Ubiquitous Computing computer science concepts Æ Ambient Intelligence miniaturization Æ Disappearing Computer energy usage battery technique Æ Invisible Computing display technologies price ... F. Ma. 36

Friedemann Mattern

Outline

4 Technology Trends The Vision Making Things Smart Consequences

Friedemann Mattern, ETH Zurich F. Ma. 37

Friedemann Mattern

The Vision

„In the 21st century the technology revolution will move into the everyday, the small and the invisible…“

Mark Weiser (1952 – 1999), XEROX PARC

F. Ma. 39

Friedemann Mattern

The Vision

„In the 21st century the technology revolution will move into the everyday, the small and the invisible…“

Mark Weiser (1952 – 1999), XEROX PARC

Small, lightweight, cheap, mobile processors and sensors in almost all everyday objects („embedded computing“) on your body („wearable computing“) embedded in the environment („sensor networks“) F. Ma. 40

Friedemann Mattern

The Vision

in almost all everyday objects („embedded computing“)

F. Ma. 41

Friedemann Mattern Embedded Computing Enables „Cooperating Smart Things“

Real world objects are enriched with information processing capabilities Embedded processors in everyday objects small cheap lightweight

Wireless communication spontaneous networks Sensors F. Ma. 42

Friedemann Mattern Embedded Computing Enables „Cooperating Smart Things“

Real world objects are enriched with information processing capabilities Embedded processors in everyday objects small cheap lightweight

Wireless communication spontaneous networks Sensors F. Ma. 43

Friedemann Mattern

Smart Objects

I‘m Can remember pertinent events smart! they have a memory

Show context-sensitive behavior they may have sensors Æ location / situation awareness

hello! Are responsive communicate with their environment networked with other smart objects

F. Ma. 44

Friedemann Mattern

The Vision

„In the 21st century the technology revolution will move into the everyday, the small and the invisible…“

Mark Weiser (1952 – 1999), XEROX PARC

Friedemann Mattern

The Vision

embedded in the environment („ sensor networks“) F. Ma. 46

Friedemann Mattern

Wireless Sensor Networks

Embed numerous distributed devices to monitor the physical world

F. Ma. 47

Friedemann Mattern

Wireless Sensor Networks

Embed numerous distributed devices to monitor the physical world

Network these devices so that they can coordinate to perform higher-level tasks

F. Ma. 48

Friedemann Mattern

Wireless Sensor Networks

Embed numerous distributed devices to monitor the physical world

Network these devices so that they can coordinate to perform higher-level tasks Combine sensing, communication and computation into a complete architecture possible by advances in low power wireless communication technology MEMS bringing rich array of cheap, tiny sensors

F. Ma. 49

Friedemann Mattern Sensor Net Applications – Biological Monitoring

Long-rangeLong-range RadioRadio

image source: Michael Scott, Philippe Bonnet

http://birds.cornell.edu/ F. Ma. 56

Friedemann Mattern

Sensor Networks – Vision

Dense instrumentation by massively distributed networks of tiny processing elements computationally-augmented environments („smart spaces“) environmental, in situ monitoring surveillance and security military Monitoring is the emergency analysis generic „killer traffic monitoring application“ medical monitoring condition-based maintenance SF smart paint? ingestible device networks?

F. Ma. 57

Friedemann Mattern

Challenges

Small-form-factor nodes, densely distributed to achieve physical locality to sensed phenomena Design constraints energy bandwidth node size and cost robustness, node and link reliability Devices must adapt automatically to the environment Berkeley COTS too many devices for manual configuration smart dust motes environmental conditions are unpredictable

F. Ma. 58

Friedemann Mattern

Challenges – Massive Parallelism

Large numbers of unattended device individual devices are not important Tolerate device failures and irregularity exploiting redundancy graceful system degradation Self-organization ad-hoc healing clustering, configuration

team formation image source: routing David Culler Information aggregation, in-network processing OS, middleware, infrastructure, services,... F. Ma. 59

Friedemann Mattern

Outline

4 Technology Trends The Vision Making Things Smart Consequences

Friedemann Mattern, ETH Zurich F. Ma. 65

Friedemann Mattern Making Things Smart with Virtual Counterparts

Virtual world (Internet, Cyberspace) virtual counterparts

pure virt. object (e.g. email)

Real world

F. Ma. 66

Friedemann Mattern Virtual Counterparts as Artifact Memories

Virtual counterparts act as memories for 1) Aug. 3rd, 2001: …. their real artifacts 2) Aug. 5th, 2001 10:34 ….. th Updates triggered 3) Aug. 5 , 2001 10:37 ... by events 4) ... Queries from the real world return memory content

Arrived in room 564 Bayview who? Hotel where? Sensors generate events when?

10:34, Sue K. opens bag F. Ma. 68

Friedemann Mattern

Responsive Objects

An objects tells something about itself e.g., by displaying a dynamically generated homepage Content depends on cirmumstances such as context and privileges

Image source: Nokia Cf. Cooltown project (HP)

F. Ma. 69

Friedemann Mattern

The Power of Directories

Identity Directory Directory Location

Context

Who operates Direc- tory directories? WWW Who controls server information and interpretation? Internet Economic / legal / political issue?F. Ma. 70

Friedemann Mattern Bridging the Gap Between the Real World and the Virtual World?

Identify objects here

Implement all the smart behavior there

F. Ma. 71

Friedemann Mattern Virtual Worlds - It All Started with Data Processing

Data

Results - Data processing - Information processing -Simulation

- Virtual Reality F. Ma. 72

Friedemann Mattern

Narrowing the Gap

Virtual world files data bases manual data entry bar code labels Real world time

F. Ma. 73

Friedemann Mattern

Narrowing the Gap

Virtual world files data bases virtual counterparts manual data entry bar code RFID tags labels Real world time Extend the integration depth tie up smart things automatically with information systems avoid media breaks and input errors

timely information F. Ma. 74

Friedemann Mattern

Electronic Labels (RFID)

image source: Peter H. Cole F. Ma. 75

Friedemann Mattern

Electronic Labels (RFID)

image source: Peter H. Cole F. Ma. 76

Friedemann Mattern

Electronic Labels (RFID)

Identify objects from distance small IC with RF-transponder Wireless energy supply magnetic field (induction) Read and write a few 100 bits „over the air“ ~ 1 m image source: Peter H. Cole F. Ma. 77

Friedemann Mattern

Small RFID Chips

F. Ma. 78

Friedemann Mattern

The µ-Chip

image source: Hitachi F. Ma. 79

Friedemann Mattern

Load Modulation

magnetic field reader transponder energy

~ IC

data band pass change of the drain-source resistance of a FET by a binary coding signal Transponder absorbes some energy of the magn. field Turning on and off a resistor in the oscillating circuit of the transponder yields a small voltage change at the antenna of the reader typically only ~ 10 mV for a reader antenna with 100 V (i.e., 80 dB signal-“noise“ ratio) F. Ma. 81

Friedemann Mattern

Influence of Frequencies

antenna size

influence energy consumption data rates penetration of water reflection on surfaces

low medium micro wave frequency kHz MHz GHz F. Ma. 82

Friedemann Mattern

Ski Ticket

RFID tags in wrist belts

F. Ma. 84

Friedemann Mattern

RFIDs in Logistics

Product tracking Realtime inventory Fast check in and check out process unload of an entire truckload takes 30 min (instead of 150 minutes) Optimization of shelf life time

F. Ma. 87

Friedemann Mattern

RFID Applications?

„We are always very bad at predicting how a given technology will be used and for what reasons“ -- Bran Ferren, Chief Disney Imagineer

F. Ma. 91

Friedemann Mattern Smart Medical These are headache Cabinet tablets. Take at most 4 per day. Tagging of medicine packages with RFID labels Automatic content monitoring and display of related information: prescription expiry date drug recalls Optional: alerts via SMS spoken language to help blind persons to take the right medicine („talking medicine“) F. Ma. 92

Friedemann Mattern

Smart Playing Cards

Support people playing a card game by an unobtrusive smart environment playing cards equipped with RFID labels RFID antenna is placed under the table Features: count score determine winner hints for beginners cheat alarm Display: wireless PDA nearby screen Demo game „Whist“ F. Ma. 93

Friedemann Mattern Card Proxies as Virtual Counterparts

:...;;.;...,.:,.. Hi, ;;.;.. :....,.:,.. I‘m new here :...;;.;...,.:,.. ;.. :.. ;;...,.:,.. ;;.;.. :....,.:,.. ;.. ...,:.. ;;.:,.. :...;;.;...,.:,.. ;.. :.. ;;...,.:,.. ;;.;.. :....,.:,.. ;.. ...,:.. ;;.:,.. ;.. :.. ;;...,.:,.. ;.. ...,:.. ;;.:,..

F. Ma. 94

Friedemann Mattern

Cards as Personalities

What do playing cards remember? all their games? What do they communicate? How do they react to msgs?

Alice in Wonderland

F. Ma. 95

Friedemann Mattern

Cards as Personalities (2)

When are msgs created? How are msgs relayed? How do playing card proxies interact with a global (distributed?) game controller?

Î General infrastructure

Alice in Wonderland

F. Ma. 96

Friedemann Mattern

Infrastructure for Smart Things

“A Dancing Toaster” (Rich Gold, XEROX PARC) F. Ma. 97

Friedemann Mattern

Why Infrastructures at All?

Consider infrastructures in the real life examples: electricity, roads,… just there or even invisible “open platform” makes life easy (e.g., deployment of new services)

Internet infrastructure Extend the Internet to everyday objects Domain Name System (DNS registry) services: cooperating routers, time servers,... IP, TCP,…: common formats / protocols Web standards (platform for other applications)

F. Ma. 98

F. Ma. 99

Friedemann Mattern Why Infrastructure for Smart Objects? How do we organize billions of mobile smart objects that are Guarantee highly dynamic, short living,…? security privacy for applications built availability with smart objects reliability Provide services for smart objects location („where am I?“) context („are we in a meeting?“) event delivery („tell me when... happens“) brokering („find something that…“) directory registry … F. Ma. 100

Friedemann Mattern

More Infrastructure Tasks

for communities Enable of smart objects spontaneous networking cooperation among smart objects communication mobility Challenge for practical computer science research! service creation service discovery (“is a service available that ...?”) ...

Facilitate linking the real world to the virtual world

F. Ma. 101

Friedemann Mattern

Outline

4 Technology Trends The Vision Making Things Smart Consequences

Friedemann Mattern, ETH Zurich F. Ma. 103

Friedemann Mattern

Economic Impact of Ubicomp?

Products that are fully integrated in information systems e.g., supply chain optimization

New digitally enhanced products e.g., cooperating toys,...

New services („e-utilities“) e.g., management of smart devices at home, management of personal privacy,...

F. Ma. 105

Friedemann Mattern

Imagine an „Internet of Things“

Detailed and timely knowledge of product location and life cycles, individual and dynamic prices for goods,... higher taxes if product is transported by plane milk bottle reduces its price with its age

F. Ma. 106

Friedemann Mattern

Imagine an „Internet of Things“

F. Ma. 107

Friedemann Mattern

Privacy in a Ubicomp World?

Privacy is already a concern with the WWW what do they do with my personal data? are my page visits and mouse clicks analyzed?

Much more dramatic in a ubicomp world! many events of very elementary actions are registered could be assembled to perfect profiles

- information fusion Bought on 20 Aug 2001; last travel: to London Sep 2003; contained shirt no. - data mining 1342 and 1349; was in Hotel Atlantic, - search engines room 317 on 17 Nov 2002 ...

F. Ma. 108

Friedemann Mattern

Privacy Challenges

Unlimited coverage (sensors everywhere) Loss of awareness („invisible computing“) New types of data (location, health, habits, …) More knowledge through context Anonymity hard to achieve Explicit notice, consent by user difficult ...

And what about trust? trusting smart everyday things? F. Ma. 109

Friedemann Mattern Ron Rivest: The Digital Revolution Reverses Defaults

1001110100011111001100111000111100000001111100111 0001001110100011111001100111000111100000001111100 1110100011111001100111000111100000001111100111011 What was once private 1000111110011001110001111000000011111001110000001is now public 1010001111100110011100011110000000111110011100111 What was once hard to copy 1000111110011001110001111000000011111001110000000is now trivial to duplicate 0011101000111110011001110001111000000011111001110 What was once forgotten 0100011111001100111000111100000001111100111011110is now stored forever 1111010001111100110011100011110000000111110011111 1010001111100110011100011110000000111110011100110 0011101000111110011001110001111000000011111001110 0001001110100011111001100111000111100000001111100F. Ma. 110

Friedemann Mattern General Impact: Evolution vs. Revolution

Technology and science have a major impact on our society and the world we live historically: industrialization, electricity, trains and automo- Performance biles, electronic mass media „revolutio- nary“ new implies therefore eventually application also ethical questions domains Time social adaptation to technical impacts needs some time since Impact this is an evolutionary process (willingness to learn, generational aspects,…)

F. Ma. 111

Friedemann Mattern

Challenges for Computer Science

„Computing without computers“ information processing moves to the background

Increasing importance of computer science computerizing and networking „all“ objects computer science has competence in organizational issues of complex, dynamic information spaces object orientation, knowledge representation, semantics,... good engineering principles are important

F. Ma. 112

Friedemann Mattern

Challenges for Computer Science

Computer Science is about architectures for information spaces and virtual worlds

F. Ma. 113

Friedemann Mattern Algorithms? Concepts? Models? Distributed Computing Theory?

F. Ma. 115

Friedemann Mattern Algorithms? Concepts? Models? Distributed Computing Theory?

In theory, there is no difference between theory and practice. But, in practice, there is. -- Jan L. A. van de Snepscheut

F. Ma. 116

Friedemann Mattern Algorithms? Concepts? Models? Distributed Computing Theory?

In theory, there is no difference between theory and practice. But, in practice, there is. -- Jan L. A. van de Snepscheut

What are adequate models, concepts and abstractions for ubicomp? that help to understand the relevant issues and help to solve problems in practice?

F. Ma. 117

Friedemann Mattern

Routing

Networks typically are dynamically changing wireless & mobile ad hoc (& peer-to-peer) d k dense, largely populated ( 3 ) d k ( 3 ) heterogeneous, ... d k k ( 3 ) d Multi-hop routing Attenuation much more energy exponent efficient than single hop! k = 2 ... 4 Dynamic routing considering failures, geographic position,... Global fair use of energy optimization (e.g., minimum power topology), game theory,... F. Ma. 118

Friedemann Mattern

Communication

Event-based, publish/subscribe (i.e., producer- consumer) vs. client-server and peer-to-peer decoupled operation in time and space, anonymous, stateless Context-aware and location-aware protocols e.g., geographic multicast Power-aware protocols Spread of broadcast information in geographically dense networks sparsely connected substructures „tolerant“ to small network changes F. Ma. 119

Friedemann Mattern

Adaptability

Adapt to changing contexts available resources discovery services (resources, equipment, capabilities) unpredictable disconnections, spontaneous and unanticipated interactions, rapidly changing topologies Location, proximity high volatility

F. Ma. 120

Friedemann Mattern

Availability, Reliability, QoS

Fault tolerance, robustness abundance of failure-prone nodes connectivity guarantees Dynamic reconfigurations fast, minimal effort Real-time performance

F. Ma. 121

Friedemann Mattern

Security and Privacy

Trust, confidence in an „open“, highly dynamic world? trust models? Physical proximity as an enabling concept? but what, exactly, is proximity? Authentication, delegation of rights fully automatic? accountability? Power-awareness e.g., cryptographic protocols

F. Ma. 122

Friedemann Mattern

Conclusions

Ubiquitous computing technologies will have a major impact on our society and the world we live Whole world as a distributed system very large, highly dynamic, open real and virtual world closely connected Challenges basic technologies, technical infrastructure concepts, models, algorithms image: EU Disappearing Computer The Internet only connected computers, Initiative now we begin to network all things F. Ma. 124

Ubiquitous Computing

Friedemann Mattern ETH Zurich Institute for Pervasive Computing

Eidgenössische Technische Hochschule Porquerolles May 2003 EETTH Zürich