Products as Agents: Metaphors for Designing the Products of the IoT Age Nazli Cila1, Iskander Smit2, Elisa Giaccardi3, Ben Kröse1 1 Amsterdam University of Applied Sciences, Digital Life Centre, Amsterdam, The Netherlands, [email protected], [email protected] 2 Info.nl, Amsterdam, The Netherlands, [email protected] 3 Delft University of Technology, Department of Industrial Design, Delft, The Netherlands, [email protected]

ABSTRACT empower people and enrich their everyday life. In order to Design-based inquiries into the networked products of the gain insights on how to create networked products that (IoT) lack a coherent understanding of attain these aims, as design researchers, we are required to the effect of such products on society. This paper proposes better understand not only the technical infrastructure and a new taxonomy for networked products, which would technological parameters of networked products (as IoT is allow articulation on their current state and future, and generally tackled in engineering and computer sciences), provide insights to designers for creating meaningful and but also the social relationships of these products with aesthetic products of IoT. Central to this framework is the everyday practices of people [37]. proposition that our current product-scape should be understood as a distribution of material agencies and best For the past couple of decades, there has been a shift in the analyzed through the metaphor of “agency”. We identify definition of designers’ main task from designing “things”, three types of agencies, i.e., the Collector, the Actor, and which are objects, to designing Things, namely socio- the Creator, and discuss how this approach could create material assemblies [6,7]. Things have become political new design methodologies to create more meaningful gatherings around shared matters of concern with a visible networked products that would empower people in their effect in the world. When it comes to the current product- everyday lives. scape of IoT, however, we fail to observe this Thing-ness. This situation is humorously demonstrated at the blog “we Author Keywords put a chip on it” (http://weputachipinit.tumblr.com/). Its Internet of Things; networked products; agency; metaphor; tagline “It was just a dumb thing. Then we put a chip in it. interaction design. Now it's a smart thing” accurately summarizes the current ACM Classification Keywords approach towards smart products. Clothespins that notify H.5.m. Information interfaces and presentation: you when the laundry is dry or socks that keep track of how Miscellaneous. many times they were washed indicate how shortsightedly IoT could be executed. Being smart, however, has a lot INTRODUCTION more potential. Products of the 21st century are quite different from their older brothers and sisters. Since information processing has Networked products should be a hybrid of technological become cheap and widespread, the capability to collect and developments and cultural articulation [37]. They need to handle information has become one of the many ‘materials’ be in a form that enables users to invite these products into from which products can be made [38]. Through gaining their lives and makes an impact on people’s life quality. computational power and network connectivity, cars, Design practice has been trying to invent the new IoT thermostats, and even light bulbs have begun to medium by exploring the new affordances and challenges communicate with their users, manufacturers, and of course that being connected brings [46]. Design research, one another. The speed and scale of this “Internet of therefore, should catch up with the practice by undertaking Things” (IoT) provide new design opportunities to inquiries into what these products mean for design culture and society and how to create empowering networked Permission to make digital or hard copies of all or part of this work for personal products that go beyond simply embedding a chip in or classroom use is granted without fee provided that copies are not made or something. Coming to grips with the IoT demands new distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work ways of seeing, understanding and asking pertinent owned by others than ACM must be honored. Abstracting with credit is questions about the ontological nature of smart and permitted. To copy otherwise, or republish, to post on servers or to redistribute connected products and their impact on users’ lives. to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Frameworks and theories in interaction design help making CHI 2017, May 06-11, 2017, Denver, CO, USA © 2017 ACM. ISBN 978-1- strategic choices about how to proceed and where to invest 4503-4655-9/17/05…$15.00 energy [36]. In this paper, we propose a framework that is DOI: http://dx.doi.org/10.1145/3025453.3025797 based on the metaphor of “agency” in order to do so. In our understanding, IoT currently deals with four different theoretical approach that follows from an understanding of types of products: (1) products that connect to its users to product agency by discussing the benefits of this metaphor, inform their status and receive orders, (2) products that propose design methodologies that are fitting for creating connect to its users and learn from these interactions to agents and discuss the design challenges that follow from become more intelligent, (3) products that are connected to the form and behavior of such agents. other products to exchange status information that is used to THE RISE OF AGENCY steer rule-based behavior, and (4) products that do not Scientific inquiry into the notion of agency is not new to the connect to the user or other products via Internet, but have science and technology studies, humanities, feminist studies an internal architecture that can adapt to the behavior of the and philosophy. Over the past decades, scholarship in these user. All these types indicate a capacity to sense and act disciplines has started to retire from perspectives that place autonomously. These products can learn and evolve. They human beings at its center. This non-anthropocentric can reveal new patterns and change our minds. In other understanding prompts considering human as a single knot words, they are actants with performative roles in our lives. in a system where many other nonhuman actors are also at Seeing them as agents can help us unravel the ecologies play. Each human or nonhuman actor in the system exerts between products and users, provide guidance about impact on others. analyzing and discussing the products of IoT, and eventually offer a new framework for developing The most prominent account in this regard is Actor- methodologies to design them better. Network Theory (ANT). ANT offers valuable insights into mapping the complex relationships between technologies The IoT field is no stranger to metaphors. Kuniavsky, for and humans. Scholars in this tradition revoke the privilege instance, uses the term “information shadow” to refer to the of human actor and discuss the ontological symmetry of digital information that tails a product’s usage [38]. Rose humans and nonhumans in networks of relations [40,42]. In uses the metaphor of enchantment to describe the pleasant other words, human and nonhuman are studied as equal experiences that well-designed smart products provide [49]. actors in any kind of network, and their agencies can be As casting a magic spell to an ordinary object, the smart continuously transformed into one another. This flat products gain some remarkable ability that make them more ontology is also at focus in materialist philosophies, e.g., useful, engaging and delightful, than their ordinary self. [10,13,31]. In particular, “Object-oriented ontology” Romero, Pousman and Mateas describe the potential of (OOO) represents the philosophical positions that dissociate ubiquitous computer systems to sense and react philosophy from anthropocentrism and consider objects to unpredictably to the behavior of users as an “alien live an existence that exceeds the relations with humans. presence” [48]. IoT as a field is susceptible to using Central to this proposition is considering objects “as entities descriptive metaphors, because many concepts of the field in their own right without requiring recourse to human use, are rather abstract and the effect of its outcomes are perception, or meaning making” [19]. difficult to grasp, which is what metaphors are good at overcoming. Although these metaphors vividly evoke the As a matter of fact, this is the argument that ANT and OOO invisible, yet powerfully present potential of IoT, they do stand apart. Although both perspectives ascribe human and not offer a framework in which we can systematically nonhuman equal being, in OOO the reality of objects is compare and abstract the use and impact of smart products. binary—something is either real or not regardless of the relations it enters into [14]. In ANT, on the other hand, In his seminal work, Schön introduces the term “generative alliances take the center stage and the reality of objects is metaphor” in order to broaden the extend of metaphor from defined through each object’s relation to other actors. The being a frame for looking at things to a process by which more an object enters into additional alliances and extends new perspectives on the world come into existence [52]. the range of its effect on other actants, the more real it We consider that the potential of smart products to change becomes [14]. Therefore, there is a constant dynamic behaviors of users and their capacity to make a visible transformation of things through coupling. effect on society call for a generative metaphor. A metaphor of agency can provide such new way of thinking as it Another theory from social sciences, activity theory, also accounts for the ways we currently communicate and work supports this claim. According to activity theory, with an ecosystem of responsive objects. understanding two or more actors (subject and object in activity theory terminology) is only possible through In the remainder of this paper, we will first draw selectively analyzing the relationship/activity between them—an from the perspectives on object agency coming from understanding that cannot be achieved by focusing on the various disciplines. Then, we will present a taxonomy that subject or object separately [36]. Agency, the ability to act is based on our understanding of how agency is and will be for producing effects, is a fundamental attribute of both manifested in smart products. We discuss and reinterpret subject and object. However, not every entity can be a products that fall under each type of agency in the subject. Subjects have needs that drive them for acting in taxonomy, and point out to specific issues that are raised the world, which turns the agency manifested by the subject through their material agency. Finally, we argue for a into a special character: it is the ability and also the “need” to act. For this reason, only living things can be subjects. physical world, and invisibility, i.e., having these computers Nonhuman have the ability to act but not the need to act, to operate autonomously [21]. Carrying the UbiComp which makes the relationship between the subject and the principles one step further, the (AmI) object asymmetrical. For this reason, activity theory rejects paradigm of the late nineties presented a vision on digital the symmetry granted to humans and nonhumans by ANT systems for the year 2010 and beyond [2]. AmI refers to the and OOO. In these theories nonhuman can delegate action environment-embedded electronic systems that are sensitive to human, but in activity theory delegation always flows and responsive to the presence of people, where many from human to all other kinds of agents (a detailed products cooperate seamlessly with one another to improve comparison of agencies between ANT and activity theory the user experience [1]. The word intelligence in AmI refers can be found in [36]). This theoretical position is also held to having digital surroundings exhibit specific forms of by Ingold, who considers agency to signify intentionality social interaction, such as recognizing people, personalizing [33,34]. Agency in his view is not an innate property of to their preferences, adapting themselves, and possible things but is something that emerges out of encounters with acting upon users’ behalf [1]. Recently, the concept other things or human beings and actualized in specific “human-computer integration” has been proposed to replace situations. the traditional human-computer interaction [23]. Integration is defined through a codependent partnership between the This argument advocates for a relational, emergent form of user and the product—in which partners construct meaning agency as in ANT. The relational agency provides ANT to around each other’s activities, negotiate and sometimes align with feminist theorizing in which there is an emphasis compromise—that carries the straightforward command- on the relational character of our capacities for action. and-respond way of interacting with products and systems Corresponding with the arguments of ANT and OOO, further towards a more symbiotic relationship. feminist theorist Barad also sees agency as something that occurs instead of something that one has [4]. In her view, All these perspectives emphasize how products and systems everything is entangled with everything else. Studying any have become more autonomous with the developing sort of situation where agency is displayed, that is any kind technologies. Yet, there are different ways that smart of knowledge practice, requires making an “agential cut” products could display autonomy. Below, this taxonomy between what is included and what is excluded from the will be presented. thing being studied. Barad argues that separations are PRODUCTS AS AGENTS TAXONOMY temporarily enacted so one can examine something long Products of IoT can exhibit different behaviors as agents. enough to gain knowledge about it [4]. Agential cuts are a We consider that the current trends in IoT point out to three means for thinking about complex systems and cultures, roles: the Collector, the Actor, and the Creator. Each and a critical framework for discussing what is brought behavior sketches out different aspects of the HCI design forth or ignored when analyzing such complexities. space that need further attention and calls for a different Regardless of their different viewpoints over the mode of inquiry. These roles are not meant as disconnected ontological symmetry between human and nonhuman or the categories, but rather envisioned as a scale. The degree of definition of agency, all these theories coming from product agency increases from the Collector type towards different disciplines have effectively challenged the the Creator type. Also within the same category, some traditional perspectives that restrict agency to humans. products may display more agency than the others. We will What these approaches have in common is seeing objects be using this taxonomy to compare, abstract, and generalize not solely as augmenters of human action, but as peers the current approaches and trends in HCI research, as well within a complex network [35]. In this paper, we do not as to discuss how to proceed in order to create better adopt the arguments of any of these approaches directly as networked products. our theoretical foundation but rather aim to appropriate The Collector their main message that humans and nonhumans are both (Also known as: the data reader; Used for: understanding, capable of action and making an impact on each other. This making invisible patterns visible) approach of distributed agency corresponds well with design, especially since the products have become smarter The first type of agency we can elicit from IoT has to do and more responsive. with the Collector products which sense and process information. They have the ability to aggregate data from The integration of (largely) autonomous set of interacting embedded sensors or social media platforms and feed the objects in everyday life has been explored from the early data back to its user, to other users, or to other products. days of . In his seminal work, Weiser These products are sometimes referred to as smart things (1991) envisioned a world in which computing is so [38], meta-products [32], everyware [30], or hybrid pervasive that everyday devices can sense their relationship products [31]. Most of the Collector products have a dual to us and to each other and coordinate their actions identity—a physical form and a virtual existence that is accordingly [56]. Weiser’s key objectives were ubiquity, connected to online services. Well-known self-monitoring i.e., embedding computation into the many aspects of the devices such as Jawbone or FitBit, for example, allow people to collect and utilize data on everyday activities like about the city and city life, which is utilized in the design of sleep patterns or the number of steps walked, and therefore, physical infrastructure, creation of policy, or justification of can be considered as part of this category. planning decisions. In such ways these systems create a new kind of civics, where it is not the citizens themselves, Laurel describes one of the functions of sensors as inviting but instead their mobile phones or sensors are doing the nature into collaboration [41]. When a sensor gathers civic volunteering [18]. This is a fascinating display of information about bird migrations, wind, or processes agency; the Collector products participate in social and inside a living being, the invisible patterns of nature are political exchange. brought into the realm of senses. Lapka personal environment monitor, in a similar vein, render the invisible Whether it is on personal, societal, or environmental level, radiation, electromagnetic fields, and humidity in a room the Collector products measure one or multiple parameters into abstract shapes to be displayed on a cellphone (Figure that are invisible to human perception, visualize this data on 1). Having entered our daily lives tremendously, the a screen or on the product itself, and provide handles for Collector products are not only able to tap into insight into everyday life, which has an enormous potential environmental factors, but also reveal people’s patterns of for molding users’ behavior and social practices. behavior and webs of practices. In this sense, they serve as The Actor “co-ethnographers” [28]. They have access to data and (Also known as: the interventionist; Used for: creating patterns that we as humans do not, and thus, help us see dialogs) what was previously invisible. As an example, the connected baby bottle designed by Bogers and his The second type of agency involves the Actor products, colleagues reveals the correlations between feeding quality which act autonomously according to the behaviors of users and environmental noise, formula temperature, teat size, or other products. These products sense and interpret data and feeding location [9]. During the testing of the product, like the Collector products, but also respond to it. Designers the parents welcomed these less obvious insights because create a potential space for the product behavior. The users they made certain patterns in their feeding practice visible, navigate in this space and perceive the product’s behavior which prompted the parents to make changes in their while the product is also engaged in autonomous practice to have a better experience. interpretation of the users’ behavior. Perhaps the best- known smart device, Google Nest is an Actor product in our taxonomy as it monitors users’ activities throughout the day and learns to adapt itself and the environment according to their behavior patterns. In the interactions with the Actor products, the user and the product continuously delegate action to each other. Marenko argues that this situation induces animistic responses in users [44]. The more the product seems intelligent and autonomous, the more our experience to deal with it tilts toward animism. As a matter of fact, the animistic behaviors of products can enable fluid and meaningful interactions between users and interactive systems. Van Allen and McVeigh-Schultz deliberately employed animism as a methodological framework in their design case of six interactive objects called Anithings [55]. Each Anithing has different intentions, personality, and inner life. Their exploration shows that these qualities of

Actor products have a potential to trigger myth-making Figure 1. Lapka personal environment monitor. tendencies of people and produce pleasant user-product interactions. On a larger scale, in “smart cities” where many products A similar pleasurable interaction is also experienced and apps contribute to a pulsing cloud of urban data, the through the use of the Addicted Toaster (Figure 2) [47]. As same co-ethnography is also at play. DiSalvo, Jenkins, and a behavior that some Actor products are prone to, the Lodato discuss the notion of “computed civics”, i.e., the toaster exploits online social network services to mimic situation in which civic participation emerges from sentience and gain identity. It nudges its owner to consume computation [18]. Examples include sensor systems that more toast by tweeting about it. Sterling names such count car throughput on particular roads or cycling apps material objects with immaterial identities that engage in that gather data on the quality of cyclists’ routes. The data conversations with other actors as “” [53]. Many of collected by these systems provide detailed knowledge the Actor products in fact fall under the category. Regardless of being embedded at home or outside environments, using Actor products requires coordination and negotiation. They sometimes nudge the users subtly to change their behavior, as in Bambea where the lampposts mainly “suggest” a better route, speed, or additional exercises. But sometimes the Actor products make more dramatic interventions. For example, the Addicted Toaster relocates itself to a new home if neglected or the Amazon trashcan of MIT Media Lab scan the barcodes of things thrown away and have them reordered from Amazon automatically [15]. Although the intensity and limits of the intervention could vary, what is common among the Actor Figure 2. Addicted Toaster by Simone Rebaudengo and Haque products is their intention to make a visible effect on Design+Research (Courtesy of Simone Rebaudengo). everyday life and practices. The Creator (Also known as: the self-aware; Used for: creating futures) As in Collector products, the reach of the Actor products has started to extend beyond the home environment. With The last type of agency is drawn from near future scenarios, the advancement of the notion, cities have in which the products will become the Creator of futures. become a playground to experiment with material agency as Active research is being conducted on robots that can be well. The Hello Lamppost project, for instance, uses pre- used in daily lives [12] and a robotic future that merges existing identifier codes on street furniture to enable people with everyday products [3]. This indicates that robots and sending text messages to the objects like post boxes or trash AI are breaking free from their traditional anthropometric bins (http://www.hellolamppost.co.uk/). The objects hold a looks and entering the daily lives of people. Then why not conversation with the user by passing on the information these everyday robots, or more aptly the everyday products that other residents had sent before (Figure 3). These with robotic qualities, start making a tangible difference on conversations are intended as an opportunity to share their form, the environment they are in, and the way they memories of the city and rediscover local environments. are used? While the Hello Lamppost had a playful aim and The ability to learn and evolve is a continuous concern in interaction, Bambea intends to ascribe sentience to the the robotics and AI fields. Bongard, Zykov and Lipson have lampposts in order to improve health of the residents of been developing the Starfish, which is the first robot to Amsterdam (http://digitallifecentre.nl/projecten/bambea/). It develop some sort of “self-awareness” [11]. The robot is a computing system that is composed of beacons and a synthesizes a predictive model of its own body through the smartphone app. The beacons are attached onto the interaction with its environment and uses this model to lampposts in Oost Park in Amsterdam, which have the develop new behaviors without an internal mathematical lampposts send motivational messages to the smartphones model constructed by engineers. Carrying the self- of runners in order to coach their running experience. awareness concept one step further, Samuelsen and Glette have been developing a robot system that has a connection with a 3D printer and is able to print new robots or customized robot parts instantly to tackle any situation they face [51]. As the algorithms have become this sophisticated, what kind of a future it will hold when everyday products with robotic qualities become self-aware about their form, environment, and usage? The vision of personal fabricators to expand outside of laboratories into homes has started to become a reality [27]. Connecting the self-aware everyday robots to 3D printers, as in the previous example, could open up an immense playground for displaying product agency. Since a toaster can arrange itself new host families by using the Internet (i.e., the Addicted Toaster), may be in the near future it could also order custom-made 3D printed pieces to be delivered at its owner’s home for the toaster to Figure 3. Hello Lamppost by PAN Studio, Tom Armitage and be used more efficiently. Gyorgyi Galik (Courtesy of Playable City). Such not-there-yet but feasible scenarios will not only autonomous agents. Finally, we present considerations that affect the users’ authority at home, but also influence the need to be taken into account when giving form to these nature of the design process. Devendorf and Ryokai tackle products and the potential uses of their designed behaviors. the new configurations of humans and machines in hybrid We consider that these issues could offer insights into using making [17]. They invited users to mimic a 3D printer in the full potential of smart products, as opposed to being order to elicit personal reflections on human-machine- limited with just sensor embedding. product relationship and expose tensions between agency Designing agents and control. Based on this, another configuration could be Interaction and product designers are faced with new forms the machine taking charge of improving itself or its of material affordances when creating networked products. connected products. When the nonhumans will start to These products are typically spread across various conceive such futures, designers will have to give up some platforms, technologies, and infrastructures (e.g., Google authorial control and come up with concepts and forms that Nest uses a sensor platform, an application platform, WiFi would prompt “free agents” to make choices and take network, database, and so on). This requires a holistic actions that would yield to satisfying outcomes [17]. approach during design, in which each component is Through the new forms of product agency, new roles for designed with and around to produce a coherent product designers in creation are bound to emerge. experience [37]. This integratedness needs to be present in DISCUSSION addressing the ecosystem that the product is in as well. As Users’ tendency for ascribing intelligence or intentionality argued by ANT and activity theory, studying multiple to products has been a longstanding concern in HCI. What actors means studying the relationships in between. In the is recent, however, is the realization that a product is design of smart products, instead of addressing each contextualized within a network of other products, users, product “vertically”, i.e., being only responsible for its form values, and contexts, and that it gains an agency via and behavior, a horizontal approach is needed where the establishing relationships with the actors in this network. designer is also responsible for considering the multiple This theoretical position of shared agency and the overlapping relationships with other products and contexts generative metaphor used for describing the situation open while giving form and ascribing behavior to a product. up new investigative opportunities for the design and study Although a holistic and horizontal approach that works of new forms of entanglements with smart products. across various platforms and connects various actors First, it brings in a broader perspective to adopt when applies to the design of all agents, different design methods creating a nonhuman actor, i.e., a smart object, as a may fit better to each type of agency. The main issue with component of ecologies involving complex interactions and the Collector products, for instance, is finding ways to use interrelations. Designing from an agency standpoint the collected contextual, experiential and behavioral data as requires considering the interdependence of human and creative design material in the design process—as a step nonhuman actors, and crafting meaningful interactions going further than using them solely for optimize and between all the relevant actors in a context. Second, the validate design [9]. In the design process of the Collector symmetry between human and nonhuman advocated by this products, a combination of sensor data and ethnographic perspective enables exploring new design possibilities that methods would work best in revealing the complex may be overlooked in a typical human-centered design architecture of practices and values surrounding the use of process. DiSalvo and Lukens argue that a non- these products [16]. Ethnography is well equipped to anthropocentric approach in design, when used as a position data in specific socio-cultural situations and offer deliberate exercise, allows designers to break free from the in depth insights of the use context and experiences. human form, capabilities and affordances, consider the Together, these analyses could grant the designers the ways nonhuman may figure into action and experience, and relevant quantitative data coming from the algorithms eventually see a technology with a fresh eye [19]. Third, interpreting the sensor data and the qualitative data that through the agency perspective, we can strive toward a new unveil the forms of practice the object partakes in. Such way of discussing and envisioning the use of the smart knowledge, when fed back into the iterative design process, products. It allows for a better articulation about the impact help to bridge the gap between sensor data insights and real of these products in society and discussing the ethics of experiences, and help designers to create products that are such interactions. In all these means, the agency metaphor well-fitting to the use context. extends the theoretical and social agendas of contemporary The design process of the Actor products is a viable arena HCI research. for experimenting with speculative design. These products In what follows we examine the considerations that play a have an autonomous character, and hence, a strong potential role in the design of agent products. We start by discussing to make an impact on the user and everyday practices in the potential design methods that are suitable for tackling ways that are currently unknown to us. In order imagine each type of agency. Next, we discuss the social, aesthetic, and critically reflect on these futures, the discursive space and ethical implications of delegating control to created by the speculative design would work well [43]. While doing so, a “maker” approach is also needed. new insights. This non-anthropocentric approach does not Material engagement is an important factor to care for the eliminate the human agency. It is just a shift away from critical matter at hand. The behavior of the Actor products privileging human activities and desires over other agents in should be carefully crafted considering various use order to better understand, describe, and critique a given situations and interactions. Research-through-design scenario, which eventually broadens the conditions and methods could allow for exploring not only the material issues of design and design research [19]. affordances of an Actor product, but also its effects on Delegating control socio-cultural situations. Knutsen, for example, goes Our commonsense understanding of reciprocity holds that through a “critical making” process of a playful networked the way we behave toward others affects the way others product that peaks out according to his friends’ activity at behave in return. But how does this human conduct apply to Foursquare in order to discover infrastructural landscape of our interactions with smart products that display IoT with its material and immaterial relationships [37]. This autonomous behavior? These products should know when approach, he argues, enabled him to engage with the critical to respond and when to delegate in order to provide matter of technologies differently than would have been the pleasant experiences. Failing to engage in this proper social case by studying existing products. Similarly, Rozendaal conduct can cause a growing tension between human and suggests using animated low-fidelity mockups in order to product agency. Striking examples in this regard are bring autonomous “objects with intent” to life [50]. These illustrated in the Uninvited Guests video, which is a critical mockups are invaluable to understand an object’s depiction of the tension between an elderly man and his expressiveness in relation to its purpose and reflect on its “smart” devices at home and how he deceives the system, possible implications on user activity and behavior. Overall, as would be expected from anyone whose boundaries and these RTD experiments, address the possibilities and routines are invaded by smart products alike [54]. problems implicit in the design of the Actor products and bring about a range of procedural and conceptual insights to As the smart products gain more autonomy, it becomes be articulated [25]. imperative to train these with the art of social grace and diplomacy. Let us take the talking shoe of Adidas and When it comes to the Creator type of smart products, the Google collaboration as an example (Figure 4). The main question to tackle is how designers and design sneakers have a microprocessor that translates the pressure researchers can conceive of a vision when the products start sensor and accelerometer readings into audio instructions. to use the data they collect to make physical interventions This enables the shoes literally speak with the wearer and on the way they look, move, or behave. A speculative encourage physical movement by offering pep talk or design approach, but this time using “design fiction” as a sometimes trash talk (an Actor product). Compared to method, can again be of help here. Design fiction is a FitBit or other Collector type products that track movement prototyping technique that is particularly tailored to and offer encouragement through a screen, the talking shoe facilitate studying near futures and investigate where the is quite vocal and expressive for the same purpose. ideal situation is located within the range of possible futures Although it may be easier to form an emotional bond with [43]. By not allowing the current technical and social the latter one (and possibly, being more encouraged to mechanisms to influence the discursive space, design move), the situation that one is in may not always be fictions can help to imagine how designers, users, and appropriate to deal with this shoe. That is why these Creator products can work together to create new concepts sneakers will always be in the gadget category, rather than or improvements in the current products and how this entering everyday lives as everyday products. To develop a situation will effect the nature of the design process. By meaningful interaction between users and smart products, doing so, the assumptions about the role of the designer and the context in which the interaction takes place should be his/her relationships with smart products are challenged. investigated well, together with how people in a specific Employing design fiction methods can open up new situation may respond to the product. questions and unfamiliar opportunities in this regard. Such conflicts may not only be between the user and the What all these proposed methods have in common is an product but also among products. When each product has acknowledgement of the ongoing interaction between its own goals and agendas, who will mediate the emerging human and nonhuman and looking for methods of design conflicts? Bihr and Thorne give the example of different research that give both an equal voice. A thing-centered home appliances fighting over the blinds being open or perspective can bring unique insights about the role of closed on the basis of their respective goals, such as the objects in human practices, and thus open up design coffee machine wanting them closed so the milk lasts opportunities that we may not be able to see with traditional longer, the plants wants them open, the Nest wants them user-centered design methods [28,29]. This requires a closed to keep the temperature low, and so on [5]. For now methodological re-orientation towards the concerns of the users are the ones to moderate these conflicts, but when objects in shaping how technologies are imagined, realized, there will be a time when products are able to take on more and researched [45]. Ordinary objects should be engaged in the design process as participants to collaboratively elicit negotiation among them, new challenges for interaction nature of the design process. As in any kind of design designers will arise. process, here the core values of moral epistemic standing, i.e., human welfare, privacy, universal usability, and The autonomous behaviors also raise ethical challenges to informed consent, should be followed [24]. consider how interaction design can be sensitive to the situation in which it intervenes. Fallman challenges the Form and behavior of agents question of what is ethical design in the contexts of material An understandable trap to fall when giving form to agents is objects enhanced with digital capacities [22]. He to be (unintentionally) affected by the “ultimate agent” of emphasizes the ethical and moral responsibility on the part our times—a robot. Currently the form of robots is more of the designer rather than the user. For example, designers commonly influenced by their historical and fictional should consider how people might wish to overrule the representations, based on the assumption that it is more autonomous actions performed by the smart product in natural for humans to interact with humanoid than abstract particular contexts. Expecting that a smart product would forms [3]. However, considering that the use context of be able to foresee and respond appropriately to any possible most smart products is home environment and everyday situation is a naïve idea in our current reality. Being teased life, following a normative product design methodology is a by your shoe by it asking if you are a statue would be better route to shape these products. The “robotification” of something funny when one is hanging out with friends at products should be by incorporating a clear functional home, but it could be slightly embarrassing when waiting purpose and adapting them to the domestic landscape using for the bus with strangers; and it is most probably difficult established modes of interaction, rather than simulating the for the shoe to make a differentiation between these two human form and capacities [3]. situations. Rozendaal gives the responsibility here to the A good and classic example in this regard is the designers [50]. Designers should create some space of Technological Dreams Series (Figure 5). This project is an freedom for the users, which allows a product to display an exploration of the near future where robots become autonomous action but this action could always be cohabitants in daily life and the ways they could relate with tempered or overridden by the user depending on the us [20]. The beauty of these robots, or Actor type products situation they s/he is in. in our taxonomy, comes from the abstraction: their form is adapted to accommodate brand new technologies, but still appearing familiar. A similar approach can also be seen in the aforementioned examples of the smart baby bottle or Amazon trash can: They appear as autonomous everyday products, instead of robots.

Figure 4. Talking shoe by 72andSunny, Google, and YesYesNo (Courtesy of 72andSunny and YesYesNo).

In the design of the smart products to know when to respond and when to delegate, guiding visions must be developed, which provide the means for analyzing and discussing the social, cultural, ethical, moral, ecological, Figure 5. Technological Dreams Series: No. 1, Robots by and political implications of these experiences and how Dunne & Raby (Courtesy of Fiona Raby). they foster particular relationships between users, designers, artifacts, and contexts [22]. The same ethical considerations surely also apply to the design of the Collector type of We judge sentience through actions, intentions, and products, whose promise of easier and healthier lives, and personality. Smart products display behaviors that evoke more efficient and greener cities is hinged on getting hold the perception of sentience. These kinetic behaviors of more data, or to the Creator type products which will provoke action from the user. The Caterpillar extension change the relationship between designer and user and the cable, for example, initiates a dialogue with the user about his/her energy consumption by twisting and turning as if in mappings between data and display in order to engage the pain when the TV is left on stand-by (Figure 6) [39]. The user in co-interpretation [48]. These examples demonstrate product could have exhibited other agent behaviors to how users and products can work together for making sense communicate the same message, such as Tweeting about it of patterns, and the Collector products can be used as or automatically turning off the TV, yet the Caterpillar provocateurs for reflection. employs a powerful means—by relying on the user’s CONCLUSION empathy to act upon the artificial pain s/he causes on the In this paper we have argued that IoT design and research product. Such animistic behaviors are significant for could benefit from a generative metaphor of product agency creating an emotive connection between the user and the to describe the new affordances, challenges, and product. opportunities of smart and networked products. That is, these products are active agents in shaping the network of relationships they are in together with other human and nonhuman actors. They collect data, act on data, and make visible interventions in the contexts they are in based on data, which are all powerful displays of agency. We called out three types of behaviors that smart products exhibit as agents, i.e., the Collector, the Actor, and the Creator, and described some projects that employ that sort of agency in attempt to discuss the current and potential socio-cultural impact of such products. The notion of agency, we propose, is valuable to come up with new opportunities and experiences of designing for Figure 6. The Never Hungry caterpillar by Matthias Laschke, IoT. While agency is a concept that has started to appear in Marc Hassenzahl and Sarah Diefenbach (Courtesy of the discussions in HCI to offer a pluralistic approach to Matthias Laschke). meaningful interactions between all the actors involved in a context without merely focusing on human [19, 28, 35, 36, 45], we also need inquiries into the design process of such Instead of targeting a specific goal or solving a task, the agents with regard to the relevant design methods to be physical behaviors of the products can also be designed for used, the extent of the negotiation and delegation between stimulating thinking and reflection. In the previously the agent and the user, and the forms and behaviors that are mentioned Anithings project, for example, the Actor type suitable for them, which is the discussion this paper products are designed to have a life of their own, where intended to start. The agency metaphor and its related they display unique behaviors depending on their assigned taxonomy we offer hope to suggest new theoretical, personality and knowledge [55]. Their behavior is methodological and practical directions for HCI research. expressed through a combination of visual display and sound, as well as how they influence each other. This HCI is considered to be in its “third wave” presently, that is makes the interaction with them unexpected, which is a shift from a narrow task-orientation to a broader concern intended to “foreground the meaning making role of people of improving the quality of everyday experiences [8]. Smart as they interact with an ecology of heterogeneous, tangible, products should have this mission as well. They already networked objects that behave in the world and acquire influence the rhythms and routines of our lives, which will narrative, mythic qualities in people’s lives.” potentially change our cultures, beliefs and preferences in the near future. An extended discussion of these behaviors The same approach is also sometimes experimented in and effects would facilitate enrichment and deepening of relation to the Collector type of products. The agency of the current discussion on creating user experiences with these products comes to an end after the communication of smart products, and how to provide empowerment and the gathered data. The reflection over the data and the behavior change through these agents. decision to change behaviors and practices are left for the consideration of the users. HCI field has been exploring ACKNOWLEDGMENTS with other means of data communication than the We thank Chris Speed for his invaluable insights on the aggregated numbers and stats. For example, the home subject which helped shaping some arguments in this paper. health horoscope of Gaver and his colleagues REFERENCES communicates sensor data through the language and form 1. Emile Aarts and Boris de Ruyter. 2009. New research of daily horoscopes [26]. The system embraces the perspectives on ambient intelligence. Ambient Int and ambiguity of data interpretation by shifting the Smart Envi 1, 1: 5-14. responsibility for interpretation from the system to the user. Similarly, Romero, Pousman and Mateas’s Tableau Machine is a system that deliberately avoids one-to-one 2. Emile Aarts and Stefano Marzano. 2003. The new 16. Nazli Cila, Elisa Giaccardi, Melissa Caldwell, Fionn everyday: Visions of ambient intelligence. 010 Tynan-O’Mahony, Chris Speed, and Neil Rubens. Publishing. 2015. Listening to an Everyday Kettle: How Can the 3. James Henry Auger. 2014. Living with robots: A Data Objects Collect Be Useful for Design Research?. speculative design approach. Journal of Human-Robot In Proceedings of the Participatory Innovation Interaction 3, 1: 20-42. Conference. The Hague, NL, 500-506. 4. Karen Barad. 2007. Meeting the universe halfway: 17. Laura Devendorf and Kimiko Ryokai. 2015. Being the Quantum physics and the entanglement of matter and Machine: Reconfiguring Agency and Control in meaning. Duke University Press. Hybrid Fabrication. In Proceedings of the 33rd 5. Peter Bihr and Michelle Thorne. 2016. Understanding Annual ACM Conference on Human Factors in the connected home: Thoughts on living in Computing Systems (CHI '15). ACM, New York, NY, tomorrow’s connected home. Retrieved Dec 30, 2016 USA, 2477-2486. from https://www.gitbook.com/book/connected-home- DOI: http://dx.doi.org/10.1145/2702123.2702547 book/understanding-the-connected-home/details 18. Carl DiSalvo, Tom Jenkins, and Thomas Lodato. 6. Thomas Binder, Giorgio De Michelis, Pelle Ehn, 2016. Designing Speculative Civics. In Proceedings of Giulio Jacucci, Per Linde and Ina Wagner. 2011. the 2016 CHI Conference on Human Factors in Design things. MIT Press. Computing Systems (CHI '16). ACM, New York, NY, 7. Erling Bjögvinsson, Pelle Ehn, and Per-Anders USA, 4979-4990. Hillgren. 2012. Design things and design thinking: DOI: http://dx.doi.org/10.1145/2858036.2858505 Contemporary participatory design challenges. Design 19. Carl DiSalvo and Jonathan Lukens. 2011. Non- Issues 28, 3: 101-116. anthropocentrism and the non-human in design: 8. Susanne Bødker. 2006. When second wave HCI meets Possibilities for designing new forms of engagement third wave challenges. In Proceedings of the 4th with and through technology. In From social butterfly Nordic conference on Human-computer interaction: to engaged citizen: urban informatics, social media, changing roles (NordiCHI '06), Anders Mørch, ubiquitous computing, and mobile technology to Konrad Morgan, Tone Bratteteig, Gautam Ghosh, and support citizen engagement, Marcus Foth, Laura Dag Svanaes (Eds.). ACM, New York, NY, USA, 1-8. Forlano, Christine Satchell and Martin Gibbs (eds.), DOI=http://dx.doi.org/10.1145/1182475.1182476 MIT Press, 421-437. 9. Sander Bogers, Joep Frens, Janne van Kollenburg, 20. Anthony Dunne and Fiona Raby. 2007. Technological Eva Deckers, and Caroline Hummels. 2016. Dreams Series: No.1, Robots. Retrieved Sep 20, 2016 Connected Baby Bottle: A Design Case Study from Towards a Framework for Data-Enabled Design. In http://www.dunneandraby.co.uk/content/projects/10/0 Proceedings of the 2016 ACM Conference on 21. Deborah Estrin, David Culler, Kris Pister, and Gaurav Designing Interactive Systems (DIS '16). ACM, New Sukhatme. 2002. Connecting physical world with York, NY, USA, 301-311. pervasive networks. IEEE Pervasive Computing 1, 1: DOI: http://dx.doi.org/10.1145/2901790.2901855 59-69. DOI=10.1109/MPRV.2002.993145 10. Ian Bogost. 2012. Alien phenomenology or what it’s 22. Daniel Fallman. 2011. The new good: exploring the like to be a thing. University of Minnesota Press. potential of philosophy of technology to contribute to 11. Josh Bongard, Victor Zykov and Hod Lipson. 2006. human-computer interaction. In Proceedings of the Resilient machines through continuous self-modeling. SIGCHI Conference on Human Factors in Computing Science 314, 5802: 1118-1121. Systems (CHI '11). ACM, New York, NY, USA, 1051- 12. Cynthia Breazeal and Fardad Faridi. 2016. Robot. 1060. U.S. Patent D761,895, Filed Nov 24, 2015, issued July DOI=http://dx.doi.org/10.1145/1978942.1979099 19, 2016. 23. Umer Farooq and Jonathan Grudin. 2016. Human- 13. Levi R. Bryant. 2011. The democracy of objects. Open computer integration. interactions 23, 6 (October Humanities Press. 2016), 26-32. DOI: http://dx.doi.org/10.1145/3001896 14. Levi R. Bryant. 2009. Being an Object-Oriented 24. Batya Friedman and Peter H. Kahn, Jr.. 2002. Human Ontologist and Actor-Network-Theorist is Hard! values, ethics, and design. In The human-computer Retrieved Sep 11, 2016 from interaction handbook, Julie A. Jacko and Andrew https://larvalsubjects.wordpress.com/2009/12/01/being Sears (Eds.). L. Erlbaum Associates Inc., Hillsdale, -an-object-oriented-ontologist-and-actor-network- NJ, USA 1177-1201. theorist-is-hard/ 25. William Gaver. 2012. What should we expect from 15. Susannah Cahalan. 2014. The future is now: The 10 research through design?. In Proceedings of the gadgets that will change your life. Retrieved Sep 11, SIGCHI Conference on Human Factors in Computing 2016 from http://nypost.com/2014/07/12/10-futuristic- Systems (CHI '12). ACM, New York, NY, USA, 937- gadgets-that-will-change-your-world/ 946. DOI=http://dx.doi.org/10.1145/2207676.2208538 26. William Gaver, Phoebe Sengers, Tobie Kerridge, Jonas Löwgren and Lone Malmorg (eds.). Springer, Joseph Kaye, and John Bowers. 2007. Enhancing London, UK, 251-274. ubiquitous computing with user interpretation: field 42. John Law. 2009. Actor network theory and material testing the home health horoscope. In Proceedings of semiotics. In The new Blackwell companion to social the SIGCHI Conference on Human Factors in theory, Bryan S. Turner (ed.). Wiley-Blackwell, Computing Systems (CHI '07). ACM, New York, NY, Sussex, UK, 141-158. USA, 537-546. 43. Joseph Lindley and Robert Potts. 2014. A machine DOI=http://dx.doi.org/10.1145/1240624.1240711 learning: an example of HCI prototyping with design 27. Neil Gershenfeld. 2008. Fab: the coming revolution fiction. In Proceedings of the 8th Nordic Conference on your desktop--from personal computers to personal on Human-Computer Interaction: Fun, Fast, fabrication. Basic Books, Foundational (NordiCHI '14). ACM, New York, NY, 28. Elisa Giaccardi, Chris Speed, Nazli Cila and Melissa USA, 1081-1084. L. Caldwell. 2016. Things as co-ethnographers: DOI: http://dx.doi.org/10.1145/2639189.2670281 Implications of a thing perspective for design and 44. Betti Marenko. 2014. Neo-animism and design: A new anthropology. In Design Anthropological Futures, paradigm in object theory. Design and Culture 6, 2: Rachel C. Smith, Kasper T. Vaskilde, Mette G. 219-242. Kjaersgaard, Ton Otto, Joachim Halse, Thomas 45. Bjorn Nansen, Luke van Ryn, Frank Vetere, Toni Binder (eds.). Bloomsbury Academic, 235-248. Robertson, Margot Brereton, and Paul Dourish. 2014. 29. Elisa Giaccardi, Chris Speed, and Neil Rubens. 2014. An internet of social things. In Proceedings of the Things making things: An ethnography of the 26th Australian Computer-Human Interaction impossible. Retrieved Sep 11, 2016 from Conference on Designing Futures: the Future of https://kadk.dk/co-design/research-network-design- Design (OzCHI '14). ACM, New York, NY, USA, 87- anthropology/ 96. DOI=http://dx.doi.org/10.1145/2686612.2686624 30. Adam Greenfield. 2006. Everyware: The dawning age 46. Kjetil Nordby. 2010. Conceptual designing and of ubiquiotous computing. New Riders. technology: Short-range RFID as design material. Int 31. Graham Harman. 2011. The quadruple object. Zer0 J of Design 4, 1: 29-44. books. 47. Simone Rebaudengo, Walter Aprile, and Paul 32. Wimer Hazenberg, Menno Huisman and Sara Cordoba Hekkert. 2012. Addicted products, a scenario of future Rubino. 2011. Meta products: Building the internet of interactions where products are addicted to being things. BIS Publishers. used. In Out of control: Proceedings of the 8th 33. Tim Ingold. 2013. Making: Anthropology, international conference on design and emotion, archaeology, art and architecture. Routledge. London, UK, 1-10. 34. Tim Ingold. 2011. Being Alive: Essays on movement, 48. Mario Romero, Zachary Pousman and Michael knowledge and description. Routledge. Mateas. 2007. Alien presence in the home: The design 35. Tom Jenkins, Christopher A. Le Dantec, Carl DiSalvo, of Tableau machine. Pers and Ubi Comp 12, 5: 373- Thomas Lodato, and Mariam Asad. 2016. Object- 382. Oriented Publics. In Proceedings of the 2016 CHI 49. David Rose. 2014. Enchanted objects: Design, human Conference on Human Factors in Computing Systems desire, and the Internet of things. Simon and Schuster. (CHI '16). ACM, New York, NY, USA, 827-839. 50. Marco Rozendaal. 2016. Objects with intent: a new DOI: http://dx.doi.org/10.1145/2858036.2858565 paradigm for interaction design. interactions 23, 3 36. Viktor Kaptelinin and Bonnie A. Nardi. 2006. Acting (April 2016), 62-65. with technology: Activity theory and interaction DOI=http://dx.doi.org/10.1145/2911330 design. MIT press. 51. Eivind Samuelsen and Kyrre Glette. 2015. Real-World 37. Jørn Knutsen. 2014. Uprooting products of the Reproduction of Evolved Robot Morphologies: networked city. Int J of Design 8, 1: 127-142. Automated Categorization and Evaluation. In 38. Mike Kuniavsky. 2010. Smart things: Ubiquitous European Conference on the Applications of computing user experience design. Morgan Evolutionary Computation. Springer International Kaufmann. Publishing, 771-782. 39. Matthias Laschke, Sarah Diefenbach, and Marc 52. Donald Schön. 1979. Generative metaphor: A Hassenzahl. 2011. “Annoying, but in a nice way”: An perspective on problem-setting in social policy. In inquiry into the experience of frictional feedback. Int J Metaphor and thought, Andrew Ortony (ed.). of Design 9, 2: 129-140. Cambridge University Press, Cambridge, UK, 138– 40. Bruno Latour. 2011. Networks, societies, spheres: 163. Reflections of an actor-network theorist. Int J of 53. . 2005. Shaping things. MIT Press. Communication 5, 15: 786-810. 41. Brenda Laurel. 2008. Design animism. In Re(searching) the digital Bauhaus, Thomas Binder, 54. Superflux. 2015. Uninvited guests. Video. (20 May 2015.). Retrieved Sep 17, 2016 from https://vimeo.com/128873380 55. Philip van Allen, Joshua McVeigh-Schultz, Brooklyn Brown, Hye Mi Kim, and Daniel Lara. 2013. AniThings: animism and heterogeneous multiplicity. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, New York, NY, USA, 2247-2256. DOI: http://dx.doi.org/10.1145/2468356.2468746 56. . 1991. The computer for the 21st century. Scientific American 265, 3: 94-104.