Are Children Well-Supported by Their Parents Concerning Online Privacy Risks, and Who Supports the Parents? — KOALA Project Report 2

Jun Zhao

Department of Computer Science University of Oxford arXiv:1809.10944v1 [cs.HC] 28 Sep 2018 April 28, 2018

1 KOALA Project Report 2 April 2018

Executive Summary

Tablet computers are becoming ubiquitously available at home or school for young children to comple- ment education or entertainment. However, parents of children aged 6-11 often believe that children are too young to face or comprehend online privacy issues, and often take a protective approach to restrict or monitor what children can access online, instead of discussing privacy issues with chil- dren. Parents work hard to protect their children’s online safety. However, little is known how much parents are aware of the risks associated with the implicit personal data collection by the first- or third-party companies behind the mobile ‘apps’ used by their children, and hence how well parents can safeguard their children from this kind of risks. Parents have always been playing a pivotal role in mitigating children’s interactions with digital tech- nologies — from TV to game consoles, to personal computers — but the rapidly changing technolo- gies are posing challenges for parents to keep up with. There is a pressing need to understand how much parents are aware of privacy risks concerning the use of tablets and how they are managing them for their primary school-aged young children. At the same time, we must also reach out to the children themselves, who are on the frontline of these technologies, to learn how capable they are to recognise risks and how well they are supported by their parents to cope with these risks. Therefore, in the summer of 2017, we conducted face-to-face interviews with 12 families in Oxfordshire and an online survey with 250 parents. This report summarises our key findings of these two studies.

Three key findings

1. Parents commonly associated privacy risks with access to the Internet, exposure to inap- propriate content, in-app adverts, or strangers. However, their knowledge of personal data collected by mobile apps is low. 2. Parents often think their children are too young to understand privacy risks online and delay these conversations with their children. As a result, children often rely on their par- ents’ guidance to cope with unknown risks, and they not always capable of recognising personal privacy-related risks. 3. When children do seek help from their parents, parents do not necessarily fully understand the risks themselves. Existing privacy safeguarding technologies mainly focus on enabling content control and offer little choices for raising parents’ awareness of personal data collection risks or supporting their children’s learning.

These findings provide critical input for us to consider providing better support for parents to scaffold their children’s ability of recognising and coping with online privacy risks, particularly those associated with implicit personal data tracking.

— KOALA Project Report 2 Page 1 KOALA Project Report 2 April 2018

1 Introduction

Tablet computers are widely used to complement education and provide entertainment at home and school. In the UK, 44% of children aged between 5 to 11 have their own tablet computers or use them as the major means to go online [35]. Tablet computers are used in over 70% of UK schools to complement education. A similar trend is also observed in the US, where ownership of tablets by young children has grown 5 times between 2011 and 2013 [1]. This is transforming the learning experience for young children [17, 36, 32, 38]. However, these new ways of online access pose new privacy risks to families and young children. The large number of applications (‘apps’) that can be downloaded for free are a major way in which children interact with these devices. Currently these ‘free’ apps are largely supported by monetisation of user’s personal information [2, 22]. While the use of cute characters may suggest benign practices, a large amount of personal information and online behaviour may be collected from children’s apps and shared with third party online marketing and advertising industry [37]. Such practices are not unique to children’s apps, but young children are particularly vulnerable to this type of exposure and less capable to resist the resulting (personalised) in-app advertisements and game promotions [19]. Previous work [51, 21] have discussed different types of online privacy threats that are recognisable by young children interacting with mobile devices, but personal data collection by mobile apps has not been specifically discussed. Exiting research has shown that this opaque data privacy risk is unknown to most users and extremely challenging to manage [46, 44]. Current internet safeguarding advice for young children mainly focuses on the management of content, cyberbullying or social media [19]. We argue that approaches to raising young children’s awareness of data privacy risk are under-researched, yet are critical for scaffolding children’s mental models of privacy. Our work thus focuses on understanding the current practices applied by parents for safeguarding the online privacy of their children, and identifying barriers faced by families for mediating data privacy risks effectively. We aim to explore three questions:

• What privacy risks are children aged 6-10 aware of when interacting with their tablets? • How much are parents aware of the possibility of personal data collection by the apps regularly used by their children?

• What technical and social practices are used by parents for safeguarding their young children’s online privacy, and what support do they need to mediate data privacy risks?

To achieve this, we conducted semi-structured interviews with 12 parent-child pairs from the South East area of the UK, complemented by an online survey with 220 parents from western developed countries (78.6% UK residence). Our analysis shows that current privacy tools on tablet devices provide good support for parents in our study to set up basic content control and filtering, but offer only limited capabilities for supporting them to gain awareness of data privacy risks or help their children establish this understanding. Although savvy tablet users, children in our study did not demonstrate a consistent understanding of their risk coping actions. Similarly, their parents – who are largely relied upon by young children – had limited awareness of the implicit personal data collection by third-parties or how to safeguard their children from this kind of risks. Our findings suggest that we are in need of new tools to raise parents’ awareness of the current personal data collection practices of mobile applications, and help both parents and young children to gain the skills for coping with data privacy threats on mobile platforms.

— KOALA Project Report 2 Page 2 KOALA Project Report 2 April 2018

2 Background

2.1 Children and Mobile Technology

In 2016, UK children aged 5-15 were for the first time reported to spend more time online than watching TV [35]. Tablets and smartphones are becoming the most popular devices used by children to go online, overtaking laptops or computers. However, little is known about the impact of these technologies on children’s online safety and general well-being [19]. Research has shown that teenagers and tweens have a reasonable understanding about privacy risks online and their implications, and strategies to protect themselves [11, 31], although they are fac- ing increasing challenges related to the use of Social Media platforms and smartphone devices [48]. However, younger children are generally expected to be less capable to engage with the Internet in a safe and mindful way [41, 16]. Using hypothetical scenarios and the contextual integrity frame- work, Kumar et al. [21] unpacked young children’s (aged 5-11) understandings of online privacy and identified key gaps in their recognition of how information is transmitted from/to their devices. Zhang- Kenney et al. [51] found that children aged 7-11 perceived privacy related to their use of mobile devices as being alone, hiding secrets, keeping things to yourself, or not talking to strangers, largely drawing on their offline life experiences. We extend this existing body of work and examine children’s understanding about data privacy on the mobile platforms in a contextualised and nuanced way.

2.2 Parental Mediation of Mobile Technology

Researchers have found that parents are generally concerned about their children’s online privacy risks [35, 51], and employ a range of mechanisms in an attempt to safeguard their children, such as password protections, supervising them, or setting up social practices at home (e.g. limiting access to the Internet) [35]. Our work aims to investigate whether UK parents of young children are effectively supported to choose less privacy-invasive mobile apps for their children and what parents may struggle with. A large body of previous research has investigated parents’ role in regulating their children’s’ inter- actions with technologies, such as TV or computers [25, 5, 43, 29, 42, 40]. Parents’ role in medi- ating children’s usage of the new mobile technologies are largely investigated for parents of older children [48, 49] or parents battling with young children’s screentime control [15, 14] or content fil- tering [13]. These studies highlighted the challenges that parents face with fully grasping the new technologies or socially engaging their young children. Literature suggests that parents’ engagement largely falls into three patterns [14]: active mediation, where parents take time to discuss with children about usage of technologies [28]; restrictive media- tion, where access to technologies is controlled and limited; and co-engagement, where parents con- sume technologies together with their children without injecting any critiques [13]. Existing parental mediation approaches for managing older children’s online privacy and safety have largely focused on a restriction approach, for example, controlling or monitoring teens’ interaction with their mobile phones using parental control software. However, the usability of these tools and their effectiveness has been questioned [28, 4, 48, 49]. For managing younger children’s use of mobile technologies, new solutions are developed to promote active mediation and co-engagement between parents and young children, for example, for setting up content filtering [13] or limiting smartphone use [20]. Results show that a combination of technical and social mediation can lead to a more productive learning experience and positive parent-child relationship. We extend this literature by investigating approaches currently taken by the parents to manage online privacy risks for their young children (under 11) and their effectiveness.

— KOALA Project Report 2 Page 3 KOALA Project Report 2 April 2018

2.3 Personal Data Collection on Mobile Platforms

More broadly, concerns about the collection and use of personal information have been discussed prior to the emergency of the web and smartphones. Westin [47] describes privacy as the notion of personal autonomy over what information is communicated, by whom, and when. Nissenbaum [34] views privacy as a dynamic, dialectical process of boundary negotiation, and argues that there are no universal norms: privacy preferences ought to differ according to distinct culture or context. Major smartphone operating systems like Android and iOS have endeavoured to improve on their existing privacy permission settings with the introduction of more granular controls. Despite such improvements, studies suggest that existing mechanisms are still inadequate, failing to raise users’ awareness of these settings or convey implications to their privacy risks [7, 23, 46]. In response to these problems, researchers have explored alternative, novel presentations of privacy notifications, by raising users’ awareness on the collection of sensitive personal information [3], making complex privacy notices easier to digest [18, 24], or easing the process of decision by predicting personal privacy preferences [26, 27]. More recently, we have seen a growing body of work revealing third- party data collection practices to the users in various ways, such as in an augmented permission interface [7] or through alternative visualisations [46]. The goal of these studies is to reveal the opaque personal data flows, from the mobile apps to first-party app developers as well as third- party online advertising and marketing trackers, and investigate the impact on users’ privacy mental model [44, 50, 8]. Our study explores the effectiveness of current privacy setting mechanisms for helping parents safeguard their children, and parents’ possible needs for the alternative, transparent presentation of personal data risks.

3 Methodology

We undertook a mixed method study, including qualitative semi-structured interviews and an online survey. The former allowed us to look into the practices and understandings of individual child and parent, and the latter complemented these thick descriptions with quantitative data from a larger sample. Our study was reviewed and approved by our university’s research ethics committee.

3.1 Semi-structured Interviews to Parents and Children

The parent-child interviews were designed to last no longer than one hour, and consisted of three parts. For safeguarding reasons, parents and children participants of the study stayed in the same room throughout each interview. We acknowledge that the presence of parents in the interviews may impact on the responses provided by the children. In the first part, each child participant was asked to show us two of their favourite apps or games on their tablet, which was brought along to the study. They were also asked to explain how the apps were found, e.g. by themselves or together with the parents. In this way we were able to look into both the technical and social safeguarding practice at home from the child’s point of view. The parent was then asked to comment on their privacy concerns of these favourite apps. In the second part of the study, we used three hypothetical scenarios to explore how children felt about personal data collection by the apps, and how they might cope with them. In the three scenarios the child was asked to imagine that an app was asking for access to their camera, microphone or location information and how they felt about it. We then showed each child how to check access permissions on their device, and asked how they felt about their favorite apps having access to their e.g. camera or location. At the end of this session, parents were asked to comment on how they felt about their children’s responses.

— KOALA Project Report 2 Page 4 KOALA Project Report 2 April 2018

In the final part of the interview, we showed parents and children a new way of examining possible personal data leakage of their favorite apps using a prototype application from our prior work [45] (see Figure 1). Although this platform was not specifically designed for children, we wanted to see how the explicit presentation of information transmission of mobile apps in this tool would impact participating children and parents’ mindset about data privacy risks and their existing safeguarding practices. After giving each child five minutes to explore the interface by themselves, we asked them how they felt about their data being collected by various companies. We concluded the interview by asking some open questions to parents concerning their opinions about current technology and legal support for safeguarding their children’s privacy online.

Two of children’s favorite apps

The major view examined by the participants, showing ‘where’ data is going

Figure 1: The web application used in the last phase of the lab study shows destinations to which personal data may be sent through each app, in a stacked bar chart view (left) and a map view (right). Hovering over an app (in the legend) highlights data trackers specific to that app.

3.2 Online Surveys to Parents

To confirm the themes observed from the 12 families in the lab study, we designed an online survey to parents of young children who regularly interact with mobile devices. The online survey was supported by the Prolific Academic system (http://prolific.ac), a crowdsourced online survey platform that is capable of recruiting participants fitting specific criteria. In our survey, participants were required to be parents or guardians to at least one child aged between 6 and 10, who has regular access to a tablet computer or smartphone. The survey was set up to be completed within 15 minutes, and each participant was rewarded £1.4 for completing the survey. Prolific provides comprehensive demographic information about the participants, such as their levels of family income or residential areas, which enabled us to collect data from a more diverse population than was the case in our lab study. The online survey contained 3 parts. It started with some basic demographic questions about the parent and child, such as their age, level of education and usage of devices. The parent was then asked to choose one child aged 6-10 to complete the main survey.

— KOALA Project Report 2 Page 5 KOALA Project Report 2 April 2018

There are two themes in the second/main part of the study: parents’ privacy concerns in relation to their children’s use of tablets or smartphones, and their knowledge of existing privacy control technologies, including password controls, kids app stores, and app privacy permissions. It started with generic concerns considered by parents when choosing apps for their children and then focused on their opinions to specific personal data collection behaviours, such as accessing cameras, location information or sensitive personal information. Finally the survey was concluded with a reflection section, asking whether the parents felt they learnt anything new about safeguarding their children’s privacy and whether they would communicate this new knowledge to their child.

3.3 Participants

Our lab study participants were recruited through social media, public posters, mailing lists, and snowball sampling. All interviews were conducted in person, and parents and children stayed in the same room throughout. The interview protocol was designed to take 30-45 minutes. Each participant child received a £15 Amazon Gift Voucher as a token of appreciation. In total we had 12 families and 14 children, with at least one child from each family aged between 6 and 10 (inclusive). Mean age of the children was 7.7. More mothers (10 participants) than fathers accompanied the child to the study. Siblings were included in the same interview if they were both within the age of 6-10. The online survey received 250 responses. 29 responses were excluded because their children were outside the age range of 6 and 10. Of the remaining 221 responses, the average age of the parents was 35.9, and 69.5% were female. Most of our respondents (78%) were UK residents, 15% US residents, and the remaining 7% reside in other western countries. The UK residents were evenly distributed in different regions of the country. The respondents had on average 2.27 children (range 1-5, SD = 0.88), and the average age of the child selected for completing the survey was 7.91 (range 6-10, SD = 1.41).

3.4 Data Analysis Method

We collected 10.5 hours of interview audio in total. The audio files were carefully transcribed and Thematic Analysis [9] was applied by 2 researchers to identify common themes. Thematic Analysis allows us to take a ground-up approach to data analysis, without being confined to a prior theoretical framework. The two researchers started by coding half of the transcripts independently. We then discussed the codes that emerged, consolidated them, and organised them into 5 themes: children’s understanding, children’s coping strategies, parents’ concerns, parental mediation methods and par- ents’ expectations of support. Both researchers used the new codes to code all the transcripts and differences were discussed and resolved. The survey data was mostly analysed using a quantita- tive data analysis approach, although the three free-text questions were analysed using Thematic Analysis following the same process as above.

4 Findings

In what follows, we first describe how our parents perceive and manage risks. We then outline chil- dren’s understandings of and coping abilities with online privacy risks. Finally, we highlight parents’ needs for managing their children’s data privacy risks related to their children’s use of mobile devices.

— KOALA Project Report 2 Page 6 KOALA Project Report 2 April 2018

4.1 Parents’ Concerns

In both the lab study and online survey, parents said that they were generally very concerned of their children’s use of these devices: only 2 families in the lab study mentioned that they never thought about privacy issues related to their children’s use of mobile devices; and 44% parents in the survey said they thought about their children’s online privacy “very often” and 31.3% said that they thought about it “sometimes”. However, our online survey data also suggests that our parents’ primary concerns were centred on app content (86.8%), followed by the cost of the apps (70%). This is consistent with parents from the lab study, as said by P1, “before privacy, I want to know what does this do?”. Furthermore, despite parents’ expressed concerns, our survey results suggest that many children in these families could still be using apps that were inappropriate for their age (i.e. content) or accessing their sensitive personal information. Table 1 shows children’s top favorite apps that were mentioned more than 5 times in the online survey responses. For example, only 4 (out of 14) of these required no access to sensitive personal data, such as contact details on the device, location information or unique identification of the device; and five of these popular apps had an age rating inappropriate for young children aged 6-10 and would require parental guidance1, including YouTube as well as some popular social media apps. The level of privacy concerns expressed by the parent participants in the survey also did not sig- nificantly correlate with the type of favorite apps used by the children: parents who were generally very concerned might still choose apps having inappropriate age rating or excessive access to their children’s personal information. In our lab study, most (10/12) parent participants expressed that they were unaware of what data about them or their children was being collected, by whom, and how it was shared with (“We have not been worried about this sharing of information before”[P2]). Only very few parents (2/14) had gone through the privacy settings on the device prior to the study (“ba- sically I will just check the permissions and [if] there is anything that I can turn it off then I will”[P1]). This indicates that there is a gap between our parent participants’ concerns and their actual level of awareness of technologies and information about apps.

4.2 Parents’ Approach to Risk Minimisation

4.2.1 Parental restrictions The main strategy to enact parental restriction by parent participants in our study was the use of password control: this was widely applied by families from both the lab study and the online survey. For example, only 24% parents reported no password control in the online survey; and 43% of them stated that they had controls of installation of even free apps, which seems to indicate a level of vigilance. This was consistent with our lab study participants, where all (but two) families had password controls for any app installations and device log-ins. Furthermore, in the lab study, we also observed that a few families sought for Internet access re- strictions. However, our interviews showed that parents did not necessarily fully understand the implications of their controls. For example, 2 (out of 12) families mentioned that they took out the SIM cards from the smartphones given to their children, and they only realised later that their children could still access to the Internet through wifi. Taken together, this suggests that restrictions might not be always effective; and even so, tend to be on the level of binary control (install/not install). The example of online connectivity restrictions highlights the challenges that parents in our study

1The information about each app is retrieved from Store, accessed in December 2017. The UK-specific interpretation of the content rating can be found https://support.google.com/googleplay/answer/6209544?hl=en-GB. PEGI 3-7 are probably more appropriate for young children aged 6-10, than those rated “requiring parental guidance”.

— KOALA Project Report 2 Page 7 KOALA Project Report 2 April 2018

App Number of use Age rating Sensitive personal data access YouTube 110 Parental guidance YES Minecraft 40 PEGI 7 YES Roblox 29 Parental guidance YES Netflix 14 Parental guidance YES CBeebies 10 PEGI 3 NO WhatsApp 10 PEGI 3 YES YouTube Kids 9 PEGI 3 YES Clash Royale 9 PEGI 7 NO Candy Crush 8 PEGI 3 NO 8 Parental guidance YES Angry Birds 6 PEGI 3 YES Pokemon Go 5 PEGI 3 YES Temple Run 5 PEGI 3 NO Music.ly 5 Parental guidance YES

Table 1: Top Favorite Apps Mentioned in the Online Survey faced in fully comprehending their choice of technologies, and hence their ability to provide sufficient protection support for their children’s online safety and privacy.

4.2.2 Passive parent-led app screening A common pattern observed in our study was reliance on parental vetting (or direct selection) of the apps to be installed. The majority of the parents (73%) in the online survey reported that they installed the apps for children after having screened them themselves and only 13% thought their children might have installed the apps by themselves. Parents from the lab study reported a higher vigilance of app screening. The majority of the parents (10/12) in the lab study believed that their children could not install any apps without their permissions (“my mum has the password, so I can’t just get it myself ” [C5]) or their awareness (“[as they use my email address], I can see what comes in”[P1]). Furthermore, some parents in the lab study would regularly go through the (“I search for free games, and categories, and then put in their ages, and type in educational”[P5]) or external recommendations to identify suitable apps for their children and install for them. Sometimes children were involved in the process; sometimes not. 2 families in the lab study mentioned that to mitigate their children’s risks, they minimised the number of apps installed on their devices. For example, P11 mentioned that

“I am always very weary with apps. For me what I would really encourage are educational apps to learn for example maths and this kind of thing”.

Although these families were generally very vigilant with their children’s access to apps, their primary focus remained on apps’ content appropriateness and controlling their children’s access. None of the above approaches would raise our parents’ awareness of personal data collection by mobile apps; nor manage these risks effectively for them.

— KOALA Project Report 2 Page 8 KOALA Project Report 2 April 2018

4.2.3 Setting home norms We found no consistent patterns in how in-home norms about app usage would be set or enforced by the parents. The majority of the parents in the lab study believed that their children would ‘naturally’ know when to ask for help, and would require their children to seek for parents’ guidance by deploying family rules or technical restrictions. However, our interviews showed that the rules were not always consistently carried out either by the parents or children. For example, C8 described that “I ask my mum and sometimes she allows it sometime not”. Despite this belief, several families from the lab study also mentioned that they required their children to use their devices in a common family area. P7, a mother of 10-year old mentioned that “We try to keep them downstairs. So they are not doing things we are not aware of ”, and P11 said that “When she is on YouTube, I try to make sure I am around. To keep an eye on her.” These observations do not suggest that family rules are ineffective, but that executing family rules against technologies can be a challenging task.

4.2.4 Active mediation sparingly used In our lab study, only 2 families explicitly talked about personal data access on mobile devices with their children. For example, when being asked how he might go and install new apps on the device, the 8-year old boy responded with the following: ... she [my mum] is strict about privacy settings, she has told me ... only apps that really need it... She doesn’t want our data to go to other people...they can break into your home” [C8]. This shows how the parent (not present in the study) was being explicit about data privacy risks with the child and trying to discuss the implications of the risks as well as setting up restrictions. Both families used simple real-world threats to help children understand the possible consequences of oversharing. No additional technologies or online resources were mentioned. We observed two themes in the rest of the families from the lab study, who did not take any active mediation approaches yet at home. One theme was that some parents believed that privacy was a concept too hard for young children and they would avoid active mediations and take a parent-led protective approach. For example, the mother of a 6-year old girl concluded that

“... she can’t understand. .. it’s my responsibility and it’s hard to expect a child to under- stand that these companies are actually manipulating their naivety”[P9]

The other theme was that a few parents were open to introducing early discussions with their children, but they struggled with communicating to their children, for example, P11 described that “This mostly comes from the school. When it comes from the parents, they don’t really [listen].”, or felt ill-equipped with their own understandings “I feel [the other parent] is more equipped to have these conversations. I barely know how to install an app.”[P5]. This suggests that the importance of actively mediating their children’s awareness from a young age is not yet widely recognised by parents in our study, and these parents showed their struggle with keeping up with the technologies or finding a way to discuss these issues with their children effectively.

4.3 Children’s Awareness of Personal Data Privacy Threats

Most parents in the online survey were not concerned about their child’s awareness of online privacy risks: 83.3% parents reported that they were “very” or “somewhat” happy, and a high proportion of them (77.7%) expected that their children would ask for help or click ’no’ to unnecessary requests for personal information access by the apps.

— KOALA Project Report 2 Page 9 KOALA Project Report 2 April 2018

However, these patterns were not supported in the lab study. Despite indications of emerging under- standing about sensitive information by our child participants, and good coping strategies from older children (above 8 years old) to directly experience-able risks, like in-app promotions or inappropri- ate content, children largely demonstrated a knowledge gap regarding personal data privacy threats related to their use of mobile devices and younger children particularly struggled. Although most of our child participants (9/14) demonstrated recognition of sensitive types of informa- tion for them, including pictures, location information, passwords, email addresses, medical records, bank details etc, the main threats perceived by them resulting from exposing their personal informa- tion would come from strangers. The majority of the children (12/14) in the lab study mentioned that they could notice risks like in- app promotions, inappropriate content, or requests to sensitive information like credit card details or passwords; and they used approaches like skipping/declining access or asking for help from parents. However, when an advert or promotion was skipped, the main reason was mainly about being bored or annoyed ("I don’t really want to watch. because I want to watch something else”[C10]). When help was sought from parents, it was more likely to be due to insufficient access to the device or a vague feeling of something not right (“...I didn’t really like the look of, so I go to my Daddy..” [C2]). This suggests that our children may not always fully comprehend the actual risks or recognise the privacy-invasive context. When we asked the child participants what they thought about one of their favorite apps requesting access to their device’s camera, location information or photos, most of the children couldn’t answer, or gave unintelligible replies, such as “I don’t know what would happen. I will press yes”[C11: a 6-yo boy]. Only a few older children (4/14) demonstrated some abilities of recognising potential risks, and associate specific social norms for different contents, like different apps, different types of personal data, or different data recipients. For example, C8, an 8-year old boy, when being asked why he thought location information was sensitive, he mentioned that “only apps that really need it, like maps and stuff ... you won’t even let the app that tracks weather and stuff to have location [because] they can break into your home”.

4.4 Children’s Coping Strategies

4.4.1 Relying on parents’ help Parents’ guidance was widely sought by our child participants to the lab study. However, we observed two themes in children’s reliance on parents: 1) a few children were consciously recognising potential risks and followed strict family norms, while 2) most children were generally motivated by a vague feeling about the situations or were restricted by what they could do on the devices by themselves. For example, a 10-year old boy [C1] recognised the risk of an app asking for access to his audio and described that “I do know when to ask...” and C8, an 8-year old boy, responded to our question about “how he would respond to an app requesting for access to his camera”, by saying that “i ask my mum ... because she is strict about privacy settings”. The majority of the children reacted passively to the risks, for example, C7 mentioned that “Dad normally test whether these things send photos”[C7]), and she would rely on Mummy to safeguard her privacy settings. This implies a demand on parents’ technical competence and also a need to raise children’s ability of recognising risky situations. It is hard to be thorough with parental controls, given the complexity of technologies. Helping children, who are in the frontier of these risks, to recognise and flag risks, is a crucial piece in safeguarding children’s privacy online.

— KOALA Project Report 2 Page 10 KOALA Project Report 2 April 2018

4.4.2 Declining access Our child participants’ ability of declining personal data access request by mobile apps correlates to their awareness about personal data collection practices of the apps. The majority of the children in the lab study showed a reasonably good ability of declining in-app game promotions or adverts. However, they were largely unable to make a conscious decision with apps asking for access to cameras or personal details. Most children had not been told about what they can/should do. If there were a “cross” button and they were annoyed and wanted to get on with their games, they would just tick it away. Only a few children demonstrated some more developed mental model regarding their decisions, which took into consideration of the needs or purposes of an app asking for access to particular information. For example, an 8-year old boy described that “I don’t want ‘chrome’ to have access to my phones... so it can show to others. and ’stickbot’ doesn’t really [need] either” [C5]. This indicates a critical gap in our child participants’ awareness and coping skills regarding personal data collection by mobile apps. This is contributed by parent participants’ own lack of awareness of the risks but also their lack of engagement with the children, believing that they are too young to understand these issues.

4.5 Parents’ Reflections on Personal Data Collection

4.5.1 Technologies hard to follow In the survey, parents expressed a high level of concerns about apps accessing their children’s location information (93%) or cameras (64%). However, less than 20% of the parents checked their children’s privacy permission settings regularly or when they installed an app. On the conclusion of the survey, nearly 86% of them felt that they learnt something through the survey and would need to go and check their children’s apps and privacy settings more carefully. This suggests that parents’ needs are largely unmet, and there is a critical gap between what parents are concerned about and what they are supported with. In our lab study 7 parent participants expressed that they found it hard to understand technologies and keep up to speed. They often relied on the other partner, who may be a Dad or Mum, to take a lead on safeguarding the children. For example, P2, a mother of a 7-year old girl, expressed at the conclusion of the interview that “he [my husband] understands more than I do. I don’t know how to do that. We keep on discussing...at some point you need to show me how to do that.”. Four parents thought they had been careful with the device controls by applying restricted access to Internet or content. However, once knowing more about the personal data privacy threats, they found it hard to be thorough, “I thought I was in a good control of what they have access to, and what they have told me. We ought to be more sensitive” [P4].

4.5.2 Feeling losing control At the conclusion of the lab interviews, the majority of the parents (9/12) expressed an increase of concerns and their surprises of not being made known. They demanded for more transparency in this space (“when you choose an app very clearly just one page just say this is what we do with your information. this is where this is going, rather than some kind of hidden secret”[P10]) or legal regulations (“Legally ... we should be protected by law. They [these companies] should simply and very clearly show all these things that they are doing”[P3]). A few parents in the lab study reacted more passively to the issue, expressing that they believed that these issues should be dealt with by relevant authorities in due course (“I am trusting other people do know, and exercising the influence and pressure” [P5]) or felt that this type of personal

— KOALA Project Report 2 Page 11 KOALA Project Report 2 April 2018 data collections was unsurprising and hard to avoid given how the current app ecosystem works (“There is nothing to stop trackers, it’s just a polite thing that you subscribe to a mailing list, but they don’t have a way to unsubscribe me”[P1]).

5 Discussions

We found that child participants in our lab study demonstrated an awareness of risks that were directly experience-able, such as in-app promotions or inappropriate content. However, they had limited knowledge about any of the more insidious personal data privacy threats and they currently largely relied on parents’ help to safeguard their online safety or provide guidance. These risks were neither fully recognised by their parents nor regularly discussed with them. Our lab study and survey results further showed that our parent participants played a key role in me- diating their children’s interactions with mobile technologies. However, our analysis of both the lab interviews and the online survey showed that our parent participants largely had limited understand- ing about personal data collection by mobile apps, and their current concerns largely centred around app content, in-app promotions and exposure to strangers online. Parent participants in both studies largely took a passive approach to safeguarding their children, by setting up technical barriers or strict family rules. Our interviews showed possible reasons behind our parent participants’ mediation approaches, including their perception that their children were too young to understand privacy risks, and their struggle of communicating with their children effectively. Existing literature has suggested that it is critical to consider active parental mediation of children’s technology usage from an early age [13, 21], when they are still interested in developing trusty relationship with grown-ups. This suggests that there might be a critical gap in exiting technologies and resources for not only rais- ing parents’ privacy awareness, but also their recognition of the importance of combining technical controls with active social interaction and mediation.

5.1 Helping Parents to Raise Their Privacy Awareness

Our parent participants’ current concerns about their children’s usage of mobile devices in terms of privacy risks are largely misplaced. Our data suggest that this is partially due to the difficulty parents expressed about keeping up with rapid developing technologies, and the lack of available tools that would be focused on raising users’ awareness of privacy risks. Privacy researchers have explored various ways to enhance the current opaque and passive privacy ‘permission’ models on mobile devices, by contextualising data access purposes [39], summarising risks [26], augmenting permission interfaces [24], and exploring just-in-time notification/nudging [3, 6]. However, in our study, we still found parents in our lab study struggling with having to scroll down to the bottom of the app page to read a piece of lengthy and jargon-laden privacy policy as their only option. This suggests that these technologies are not yet becoming accessible in the mainstream platforms. Communicating privacy risks—especially those without immediate impacts—to busy parents can be challenging. As we see, even though our parent participants in our survey largely focused on the importance of content appropriateness, their young children were still exposed to apps that were probably inappropriate to their age (see Table 1). The age rating of apps can be obscure (depending on the device and platform), and recognising apps that are indeed designed for young children in the general app store can be challenging even for trained eyes. This suggests that designing for parents may require careful consideration about information accessibility and usability. A summarised presentation of app risks [26] can provide a convenient shortcut for parents to quickly sift through

— KOALA Project Report 2 Page 12 KOALA Project Report 2 April 2018 apps with less risks. However, privacy risks can be a personal and contextualised decision making process [33], and a one-fit-all might not work for everyone’s need. Another challenge is that parents can face a more complicated social context, such as the peer pressure their children feel from their friends or siblings [51, 49]. These unique social factors could lead to either a more complex mental model or compromises to be made. Designing for parents thus needs to consider additional social context related to the technologies, such as experience comments from peers or other parents. They may provide as valuable inputs to their decision making as knowing the specific companies or jurisdictions that have access to their children’s personal data.

5.2 Filling in the Parental Mediation Void

Our study not only highlighted the knowledge gap in parent participants’ awareness of personal data privacy threats, but also the even bigger knowledge gap of their young children’s. This is alarming, given that our results showed that our child participants were already exposed to the risks, and they were unclear when to ask for help from their parents. In our study, our child participants showed a strong reliance on their parents’ help, and a strong and trusty relationship with their parents. This indicates that there might be a great opportunity for parents to take a more proactive role in facilitating children’s understanding about privacy risks. However, currently our parent participants largely believed either that their children were too young to understand or they found themselves ill-equipped to mediate their children. This suggests there might be a critical gap in modern parents’ awareness of the importance of active mediation and in tech- nologies supporting parents to take an interactive approach. This is analogous to the situation of par- enting approaches to teenage online habits. A recent review of parental control tools [48] has shown that current technologies largely focus on parental restrictions/monitoring for parents of teenagers, and provide limited support for increasing parents’ awareness or facilitating parent-child discussions, which is in fact desired by both parents and their teens. Existing studies have demonstrated that a co-learning process can facilitate a more effective mediation of technology usage between parents and young children [15, 13] Our findings suggest the need to provide more support in raising parents’ awareness on the impor- tance of discussing these issues with their children from a young age. Existing resources online are largely focused on older children (like tweens or teenagers) and do not support younger children or cover this type of emerging personal data privacy risks. Although apps or smart toys targeted for young children are currently in the limelight due to rumours about them being unsafe and prompting children to reveal personal information [12, 30], understanding of the risks associated with smart apps or smart toys is still scarce, and parents are only provided with generic cautions instead of specific advices. Technologies should also consider incentivising parents to actively discuss these issues with their young children by providing specific age options, and content specially targeted for a younger age group. Literature has suggested that parents of teenagers often face stronger push-back from their older children for interfering or mediating their use of technologies [49]. Kumar et al [21] argue that parents could miss a critical opportunity to establish a mediation relationship with their young children if they choose to delay the discussions, because “[young] children are more focused on adult/parent relationships”.

5.3 Scaffolding Children’s Understanding

The biggest challenges faced by our child participants in the lab study were that they found it hard to understand how information was used by mobile apps, who could access information and where

— KOALA Project Report 2 Page 13 KOALA Project Report 2 April 2018

information could go, and how to react to different risks, by recognising when to ask for help. This can be regarded as a complex topic for young children, and the ICT curriculum in the UK is still forming and going through rapid changes in the last few years. However, we argue that children’s early understandings could have potential long-term impact on their mental model when they grow older and have more freedom with their smartphone devices [21]. The Zone of Proximal Development (ZPD) theory from Education Psychology [10] suggests that children’s learning is better supported if education starts with individual child’s current zone of knowl- edge and supports their learning process according to their individual needs and environment. A recent report in the UK on Digital Childhood also highlighted a gradual change of focus is needed for mediating young children’s use of digital technologies according to their developmental stages [19]. The critical knowledge gaps identified in our study could provide initial indicators to future privacy education technologies. Furthermore, the web prototype we used in the last part of our lab study provided some possible future design directions for us to explore. While walking through children’s favorite apps in the pro- totype (see Figure 1), we observed that older children (aged 8-10) found it “very interesting”. They demonstrated an understanding of the information transmissions through interactions with the map view, and engaged in the discussions about the companies and countries to which their favorite apps might be sending their personal information. They responded that they believed that “it’s not right for companies from another country to access my information” [C9, aged 10] or that “[I am ] sort of surprised, but not much. because people would do anything to make more money these days” [C1, aged 10]. At the same time we observed that the application enabled an engagement between parents and their children. Parents found it a useful tool to explain to their children the implications (“now you think that these companies are kind of using these as clever tools to gather your data? you know it [means] that it’s more than a game, did you?”[P4]). This suggests that a child-friendly visusalisation of the information transmissions behind mobile apps could have the potential to not only increase children and parents’ understanding of personal data privacy risks but also provide an interactive tool for co-learning.

5.4 Limitations

The primary limitation of this study is our sample population; our lab study participants were all residents of the affluent South East area in the UK and highly educated. We fully acknowledge that this is not representative of the populations. The complementation of this interview data with an online survey that incorporated a wider and more diverse population was one step towards validating the main themes. Furthermore, we have also participated in a large-scale science outreach activity for young children in the Autumn of 2017, with >100 families participating on the day. We observed similar reactions from the children, who came from a wider part of the area. The second limitation is the lab study setting, where the presence of parents and children in the same room might influence the responses from the child participants. Furthermore, the unfamiliar setting and time constraints of the study might also limit their ability of reflective thinking. We plan to explore focus group studies with young children in future research, to reduce the influence from the inter- viewers and parents, and observe them in their familiar settings, like schools. Through this we aim to achieve a deeper understanding about the familiar terms used by the young children to describe risks and the building blocks required for them to adapt their existing knowledge to understanding new kinds of information transmission and associated risks.

— KOALA Project Report 2 Page 14 KOALA Project Report 2 April 2018

6 Conclusions

In this report we presented the results of a mixed-method study towards understanding what data privacy threats children ages 6-10 and their parents were aware of during the interaction with tablet computers. We found that although our child participants had a reasonable awareness of familiar risks, like in-app promotions or inappropriate content, they (particularly younger children) largely struggled to understand the information transmissions of mobile apps, and how their personal data might be collected and pose threats to their personal privacy. Older children in our study demon- strated some strategies of declining access to information, however, our child participants still mostly relied on parents’ help to mitigate risks. Our parents also showed limited understanding regarding what data was collected by the apps and shared with whom. Their primary privacy concerns were about content and screen time controls. To mitigate their concerns, they sought passive technical restrictions, app limitations, or external information sources. However, these tools and resources largely focused on content control and provided no support for raising parents’ awareness or facilitating discussions with their children. This is therefore a critical gap in the current landscape. Based on these findings, we recommend 1) raising the general awareness of considering scaffolding children’s knowledge about data privacy from an early age, where they are still at an age of seeking for a positive relationship with grown-ups, and are already facing risks at a regular basis; and 2) tool and resource developments that focus on facilitating skill and knowledge building for both parents and their young children, and encouraging an active co-learning experience. Our identification of children’s key knowledge gaps and feedback to an exiting prototype could provide potential inputs for the design of future privacy education technologies.

7 Selection and Participation of Children

In our ethics approved study, children were recruited through public invitations shared with parents on social media, posters in permitted public spaces, mailing lists, and snowball sampling. The study took place over school summer holiday. Thus, no recruitment took place through schools. Participants lived in the South East area of the U.K. Children were limited to the ages of between six and ten inclusively. All children from each family were encouraged to participate in the study as long as they were within the age limit and did not express distress. Parents signed an informed consent and children signed an assent form with their parents’ help (if needed). The forms explicitly requested consent for the study to be audio-recorded for the purposes of transcription.

References

[1] 2013. Zero to Eight: Children’s Media Use in America 2013. Technical Re- port. Common Sense Media. https://www.commonsensemedia.org/research/ zero-to-eight-childrens-media-use-in-america-2013 [2] Alessandro Acquisti, Curtis R Taylor, and Liad Wagman. 2016. The economics of privacy. Jour- nal of Economic Literature 52, 2 (2016). [3] Hazim Almuhimedi, Florian Schaub, Norman Sadeh, Idris Adjerid, Alessandro Acquisti, Joshua Gluck, Lorrie Faith Cranor, and Yuvraj Agarwal. 2015. Your location has been shared 5,398 times!: A field study on privacy nudging. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 787–796.

— KOALA Project Report 2 Page 15 KOALA Project Report 2 April 2018

[4] Monica Anderson, A Singh, and D Page. 2016. Parents, teens and digital monitoring. Pew Research Center: Internet, Science & Tech (2016). [5] Erica Weintraub Austin. 1993. Exploring the effects of active parental mediation of television content. Journal of Broadcasting & Electronic Media 37, 2 (1993), 147–158. [6] Rebecca Balebako, Jaeyeon Jung, Wei Lu, Lorrie Faith Cranor, and Carolyn Nguyen. 2013. Little brothers watching you: Raising awareness of data leaks on smartphones. In Proceedings of the Symposium on Usable Privacy and Security. ACM, 12. [7] Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason I Hong, and Lorrie Faith Cranor. 2014. The privacy and security behaviors of smartphone app developers. In Proceedings of Workshop on Usable Security (USEC). [8] Reuben Binns, Jun Zhao, Max Van Kleek, and Nigel Shadbolt. 2018. Measuring third party tracker power across web and mobile. Transaction on Internet Technology (2018). To appear. [9] Virginia Braun and Victoria Clarke. 2013. Successful qualitative research: A practical guide for beginners. Sage. [10] Seth Chaiklin. 2003. The zone of proximal development in VygotskyâA˘ Zs´ analysis of learning and instruction. Vygotsky’s educational theory in cultural context 1 (2003), 39–64. [11] Katie Davis and Carrie James. 2013. Tweens’ conceptions of privacy online: implications for educators. Learning, Media and Technology 38, 1 (2013), 4–25.

[12] Stuart Dredge. 2014. What the Talking Angela app is really saying to your kids. (2014). https: //www.theguardian.com/technology/2014/feb/18/talking-angela-app-children-safety Accessed in December 2017. [13] Yasmeen Hashish, Andrea Bunt, and James E Young. 2014. Involving children in content control: a collaborative and education-oriented content filtering approach. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 1797–1806. [14] Alexis Hiniker, Sarita Y Schoenebeck, and Julie A Kientz. 2016a. Not at the dinner table: Par- ents’ and children’s perspectives on family technology rules. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 1376–1389. [15] Alexis Hiniker, Hyewon Suh, Sabina Cao, and Julie A Kientz. 2016b. Screen time tantrums: how families manage screen media experiences for toddlers and preschoolers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 648–660. [16] Donell Holloway, Lelia Green, and Sonia Livingstone. 2013. Zero to eight: Young children and their internet use. (2013). http://eprints.lse.ac.uk/52630/1/Zero_to_eight.pdf [17] Sharon Judge, Kimberly Floyd, and Tara Jeffs. 2015. Using mobile media devices and apps to promote young children’s learning. In Young Children and Families in the information age. Springer, 117–131. [18] Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W Reeder. 2009. A nu- trition label for privacy. In Proceedings of Symposium on Usable Privacy and Security. ACM, 4. [19] Beeban Kidron, Anghrarad Rudkin, Miranda Wolpert, Joanna R. Adler, Andrew K. Przybylski, Elvira Perez Vallejos, Henrietta Bowden-Jones, Joshua J. Chauvin, Kathryn L. Mills, Marina Jirotka, and Julian Childs. 2017. Digital Childhood: Addressing Childhood Development Mile- stones in the Digital Environment. Technical Report. 5Rights.

— KOALA Project Report 2 Page 16 KOALA Project Report 2 April 2018

[20] Minsam Ko, Seungwoo Choi, Subin Yang, Joonwon Lee, and Uichin Lee. 2015. FamiLync: facilitating participatory parental mediation of adolescents’ smartphone use. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 867–878. [21] Priya Kumar, Shalmali Milind naik, Utkarsha Ramesh Devkar, Marshini Chetty, Tamara L. Clegg, and Jessica Vitak. 2018. No Telling Passcodes Out Because They’re Private?: Understand- ing Children’s Mental Models of Privacy and Security Online. In Proceedings of ACM Human- Computer Interaction (CSCW ’18 Online First). ACM. [22] Michael E Kummer and Patrick Schulte. 2016. When private information settles the bill: Money and privacy in Google’s market for smartphone applications. ZEW-Centre for European Eco- nomic Research Discussion Paper 16-031 (2016).

[23] Ilaria Liccardi, Joseph Pato, and Daniel J. Weitzner. 2014a. Improving Mobile App Selection through Transparency and Better Permission Analysis. Journal of Privacy and Confidentiality: Vol. 5: Iss. 2, Article 1. (2014), 1–55. [24] Ilaria Liccardi, Joseph Pato, Daniel J Weitzner, Hal Abelson, and David De Roure. 2014b. No technical understanding required: Helping users make informed choices about access to their personal data. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ICST (Institute for Computer Sciences, Social- Informatics and Telecommunications Engineering), 140–150. [25] Carolyn A Lin and David J Atkin. 1989. Parental mediation and rulemaking for adolescent use of television and VCRs. Journal of Broadcasting & Electronic Media 33, 1 (1989), 53–67.

[26] Jialiu Lin, Shahriyar Amini, Jason I Hong, Norman Sadeh, Janne Lindqvist, and Joy Zhang. 2012. Expectation and purpose: understanding users’ mental models of mobile app privacy through crowdsourcing. In Proceedings of Conference on Ubiquitous Computing. ACM, 501– 510.

[27] Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, Shikun Aerin Zhang, Norman Sadeh, Yuvraj Agarwal, and Alessandro Acquisti. 2016. Follow My Recommendations: A Personalized Assistant for Mobile App Permissions. In Proceedings of the Symposium on Usable Privacy and Security. [28] Sonia Livingstone and Ellen J Helsper. 2008. Parental mediation of children’s internet use. Journal of broadcasting & electronic media 52, 4 (2008), 581–599. [29] May O Lwin, Andrea JS Stanaland, and Anthony D Miyazaki. 2008. Protecting children’s privacy online: How parental mediation strategies affect website safeguard effectiveness. Journal of Retailing 84, 2 (2008), 205–217. [30] Emily McReynolds, Sarah Hubbard, Timothy Lau, Aditya Saraf, Maya Cakmak, and Franziska Roesner. 2017. Toys that Listen: A Study of Parents, Children, and Internet-Connected Toys. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5197–5207. [31] Lien Mostmans, Joke Bauwens, and Jo Pierson. 2014. “I would never post that”: Children, moral sensitivity and online disclosure. Communications 39, 3 (2014), 347–367.

[32] Michelle M Neumann and David L Neumann. 2014. Touch screen tablets and emergent literacy. Early Childhood Education Journal 42, 4 (2014), 231–239.

— KOALA Project Report 2 Page 17 KOALA Project Report 2 April 2018

[33] Janni Nielsen, Torkil Clemmensen, and Carsten Yssing. 2002. Getting access to what goes on in people’s heads?: reflections on the think-aloud technique. In Proceedings of the Nordic conference on Human-computer interaction. ACM, 101–110. [34] Helen Nissenbaum. 2004. Privacy as contextual integrity. Wash. L. Rev. 79 (2004), 119. [35] ofcom.org. 2016. Children and parents: media use and attitude report. (2016). https://www.ofcom.org.uk/research-and-data/media-literacy-research/childrens/ children-parents-nov16 [36] Stamatis Papadakis, Michail Kalogiannakis, and Nicholas Zaranis. 2016. Comparing tablets and PCs in teaching mathematics: an attempt to improve mathematics competence in early childhood education. Preschool Prim. Educ 4, 2 (2016), 241–253.

[37] Irwin Reyes, Primal Wiesekera, Abbas Razaghpanah, Joel Reardon, Narseo Vallina-Rodriguez, Serge Egelman, and Christian Kreibich. 2017. “Is Our Children’s Apps Learning?” Automatically Detecting COPPA Violations. In Workshop on Technology and Consumer Protection (ConPro 2017), in conjunction with the 38th IEEE Symposium on Security and Privacy. [38] John Schacter and Booil Jo. 2016. Improving low-income preschoolers mathematics achieve- ment with Math Shelf, a preschool tablet computer curriculum. Computers in Human Behavior 55 (2016), 223–229. [39] Fuming Shih, Ilaria Liccardi, and Daniel Weitzner. 2015. Privacy tipping points in smartphones privacy preferences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 807–816.

[40] Wonsun Shin and Jisu Huh. 2011. Parental mediation of teenagers? video game playing: Antecedents and consequences. New Media & Society 13, 6 (2011), 945–962. [41] Benjamin Shmueli and Ayelet Blecher-Prigat. 2010. Privacy for children. Colum. Hum. Rts. L. Rev. 42 (2010), 759.

[42] Martin Valcke, Sarah Bonte, Bram De Wever, and Isabel Rots. 2010. Internet parenting styles and the impact on Internet use of primary school children. Computers & Education 55, 2 (2010), 454–464. [43] Patti M Valkenburg, Marina Krcmar, Allerd L Peeters, and Nies M Marseille. 1999. Developing a scale to assess three styles of television mediation:?Instructive mediation,??restrictive me- diation,? and ?social coviewing? Journal of broadcasting & electronic media 43, 1 (1999), 52–66. [44] Narseo Vallina-Rodriguez, Srikanth Sundaresan, Abbas Razaghpanah, Rishab Nithyanand, Mark Allman, Christian Kreibich, and Phillipa Gill. 2016. Tracking the Trackers: Towards Un- derstanding the Mobile Advertising and Tracking Ecosystem. arXiv preprint arXiv:1609.07190 (2016). [45] Max Van Kleek, Reuben Binns, Jun Zhao, and Nigel Shadbolt. X-Ray Refine: Supporting the Exploration and Refinement of Information Exposure Resulting from Smartphone Apps. In Pro- ceedings of 2018 CHI Conference on Human Factors in Computing Systems. to appear. [46] Max Van Kleek, Ilaria Liccardi, Reuben Binns, Jun Zhao, Daniel J Weitzner, and Nigel Shadbolt. 2017. Better the devil you know: Exposing the data sharing practices of smartphone apps. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5208–5220.

— KOALA Project Report 2 Page 18 KOALA Project Report 2 April 2018

[47] Alan F Westin. 1968. Privacy and freedom. Washington and Lee Law Review 25, 1 (1968), 166. [48] Pamela Wisniewski, Arup Kumar Ghosh, Heng Xu, Mary Beth Rosson, and John M Carroll. 2017a. Parental Control vs. Teen Self-Regulation: Is there a middle ground for mobile online safety?. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 51–69.

[49] Pamela J Wisniewski, Heng Xu, Mary Beth Rosson, and John M Carroll. 2017b. Parents Just Don’t Understand: Why Teens Don’t Talk to Parents about Their Online Risk Experiences.. In CSCW. 523–540. [50] Zhonghao Yu, Sam Macbeth, Konark Modi, and Josep M Pujol. 2016. Tracking the Trackers. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 121–132. [51] Leah Zhang-Kennedy, Christine Mekhail, Yomna Abdelaziz, and Sonia Chiasson. 2016. From Nosy Little Brothers to Stranger-Danger: Children and Parents’ Perception of Mobile Threats. In Proceedings of the The 15th International Conference on Interaction Design and Children. ACM, 388–399.

— KOALA Project Report 2 Page 19