Aggression and Violent Behavior 48 (2019) 169–179

Contents lists available at ScienceDirect

Aggression and Violent Behavior

journal homepage: www.elsevier.com/locate/aggviobeh

Consistently inconsistent: A systematic review of the measurement of use T ⁎ Ethan A. Marshall , Holly A. Miller

Department of Criminal Justice and Criminology - College of Criminal Justice, Sam Houston State University, United States of America

ARTICLE INFO ABSTRACT

Keywords: Research indicates that pornography use is now practically ubiquitous among males and continually increasing Pornography among females. These statistics are concerning in light of decades of research signaling that pornography use Sexual aggression may be associated with sexually coercive behavior. Though the relationship between pornography use and Sexual coercion sexual coercion has been a focus of concern, the apparent inconsistency in methods used to assess pornography Measurement limits the field from approaching a consensus on the strength of this relationship, as well as developing a Assessment thorough understanding regarding which aspects of pornography use drive this relationship. The purpose of the current study is to systematically review the literature on pornography use over the last ten years. This review will provide an updated examination of the operationalization and assessment of pornography use in peer- reviewed studies, synthesize the assessment of pornography use from various disciplines, and provide sugges- tions for the assessment of pornography use moving forward. Results of this review will serve as a potential guideline for the improvement of methodologies used to assess pornography use, and to facilitate movement towards more consistent methodological approaches to strengthen research examining the relationship between pornography use and sexual coercion.

1. Introduction pornography consumption and sexually coercive behaviors. This can only be accomplished, however, through accurate and effective mea- Over the last several decades, researchers have attempted to un- surement of pornography use. derstand whether pornography use influences human behavior, and if Early studies on pornography use and sexual coercion were typically so, what types of behavior are influenced by this practice. While some conducted in laboratories (Allen, D'Alessio, & Brezgel, 1995a; Allen, researchers highlight the potential benefits pornography consumption Emmers, Gebhardt, & Giery, 1995b), where participants would be ex- has on sexual satisfaction and functioning, one common area of interest posed to pornographic material in a laboratory setting, and their scores in pornography use and sexual behavior focuses on the relationship on various attitudinal measures were compared with individuals who between pornography use and sexual coercion. Interest in this area of were not exposed to pornographic material. In the last few decades, study has garnered amplified attention with the increasingly easier however, researchers have shifted to self-report survey methods for access to pornography via the . Recent statistics indicate that examining this relationship. These types of studies have yielded a large the overwhelming majority of individuals in Western society, both male amount of support for the relationship between pornography use and and female, have been exposed to some type of pornographic material sexually coercive behaviors, as indicated by a meta-analytic study by in their lifetime (Brown & L'Engle, 2009; Mossige, Ainsaar, & Svedin, Wright, Tokunaga, and Kraus (2016), but these studies provide only 2007; Romito & Beltramini, 2011; Sabina, Wolak, & Finkelhor, 2008; that there is an association, not a linear relationship, between porno- Wolak, Mitchell, & Finkelhor, 2007). Now that this type of material is graphy use and sexually coercive behaviors, and do very little to explain available to virtually anyone, anywhere, at any time, it has resulted in a how and why pornography use is predictive of sexual coercion. Re- massive influx of individuals viewing pornography, including children, search findings, however, have provided some specific aspects of por- and a significant increase in the amount of pornography consumed nography use that warrant further examination. For instance, studies (Brown & L'Engle, 2009; Mossige et al., 2007; Sabina et al., 2008; have shown that use of violent pornography is significantly associated Wolak et al., 2007). With this increase in use, there comes an even with sexually coercive behaviors (Kingston, Fedoroff, Firestone, Curry, greater need to determine what drives the relationship between & Bradford, 2008; Wright, Sun, Steffen, & Tokunaga, 2015). The

⁎ Corresponding author. E-mail address: [email protected] (E.A. Marshall). https://doi.org/10.1016/j.avb.2019.08.019 Received 25 January 2019; Received in revised form 2 June 2019; Accepted 24 August 2019 Available online 29 August 2019 1359-1789/ © 2019 Elsevier Ltd. All rights reserved. E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179 immediacy of this finding is made salient in light of research indicating purpose of the study, this did limit their review from capturing how the heightened aggressive content of even “mainstream” pornography researchers involved in lines of inquiry other than pornography and (Bridges, Wosnitzer, Scharrer, Sun, & Liberman, 2010). Researchers sexual coercion were operationalizing pornography use and excluded a have also highlighted habitual characteristics other than frequency of large body of research examining the relationship between porno- use that may factor into the relationship between pornography and graphy use and sexual coercion in offender populations. This establishes sexual coercion. These other habitual factors include number of mod- the importance of the current study, which seeks to improve upon the alities used to view pornography (Marshall, Miller, & Bouffard, 2017, methods used by Short et al. (2012), so several unique recommenda- 2018) and age at first exposure (Burton, Leibowitz, & Howard, 2010; tions to the current methodologies of research on pornography use can Mancini, Reckdenwald, & Beauregard, 2012). Other studies highlight be provided. the importance of sexual scripts adopted from pornography use First, the current study will include a wider variety of disciplines in (Marshall et al., 2017; Tomaszewska & Krahe, 2018). Sexual scripts are the search for studies, in order to capture a broader idea of how various conventions used to inform sexual desire, interpersonal sexual situa- disciplines are operationalizing pornography use. Findings from this tions, and sexual behavior through their interpretation. Researchers study will not only provide an updated assessment of the consistency, or believe that pornography use may work through sexual scripts to in- lack thereof, in pornography use research, it will pull approaches from fluence sexual behavior, and more specifically, sexual coercion. Finally, other disciplines that may give insight into how to better operationalize Marshall et al. (2017) have begun examining whether there are and assess pornography use to improve our understanding of the re- thresholds of pornography use beyond which use becomes predictive of lationship between pornography use and sexual coercion. Second, over sexual coercion, and provided evidence of the existence of these eight years have passed since Short et al. (2012) conducted their initial thresholds. systematic review. In this time, smart phones have become nearly Although researchers have established a large body of literature on universal for adults and young people, and recent research indicates the relationship between pornography use and sexual coercion, there that these devices are used for pornography consumption as much as are inconsistencies across these studies. A systematic review focusing desktop or laptop computers (Zheng, Zhang, & Feng, 2017). With stu- on the provided definitions of what constitutes “pornographic material” dies indicating that smart phones are now a popular method for viewing and how scholars operationalized the concept of pornography use, pornography, it is important to examine whether researchers are provided evidence of these issues (Short, Black, Smith, Wetterneck, & keeping up with these recent developments. Finally, the current sys- Wells, 2012). First, the review revealed that, in the rare instance that tematic review not only seeks to assess the current state of methods the researcher provided a definition of pornography to respondents, the used to assess pornography use, but to also offer suggestions regarding definitions varied widely. Additionally, inconsistencies in the mea- assessment of pornography use to researchers interested in examining surement of pornography were revealed. Some of these may be some- pornography use and sexual coercion. Short et al. (2012) identified the what trivial, with response categories for frequency of use in one study importance of establishing some sort of validated pornography use as- ranging from “never” to “very often” and from “never” to “very fre- sessment, and one way this can be accomplished, is by examining the quently” in another. Some studies, however, ask the respondents to latest research from the various fields that take interest in this ever- recall the number of days over the last year that they viewed porno- increasing human behavior. graphy (Kingston et al., 2008), while others require respondents to select from anchors indicating “monthly,”“weekly,” or “daily” use 2. Methods (Marshall et al., 2017, 2018) and simple dichotomized variables (Luder et al., 2011). Furthermore, the majority of these studies accounted only The strategy used to search for and identify articles in the current for frequency of use, ignoring many of the other aspects shown to be review was similar to the systematic review conducted by Short et al. important - like violent pornography use - in examining the relationship (2012). First, search terms were developed. For the current study, between pornography consumption and sexual coercion. Short et al. “pornography,”“sexually explicit material,” and “sexually explicit (2012) also indicated that nearly all of the studies were conducted using media” were employed. These search terms provide a slight expansion researcher-generated questions, highlighting the need for a standar- from the terms provided by Short et al. (2012), who used “internet dized assessment of pornography use. In summation, the researchers pornography” and “sexually explicit material.” Additionally, a filter stated “this review reveals a lack of consistency in was applied to the search excluding all articles that were published research, complicating comparisons across studies and possibly hin- more than 10 years preceding the date of the current study (July 2018). dering future research in this area” (Short et al., 2012; p. 22). These search terms were then used to identify articles on the “So- This lack of consistency demonstrated by Short et al. (2012) is cIndex,”“PsycInfo,” and “Criminal Justice Abstracts” databases. These problematic for several reasons. First, meta-analyses, such as the one searches yielded a total of 5692 articles. After excluding duplicates, produced by Wright et al. (2016), must be conducted and interpreted 4510 articles remained. The abstracts for all of these articles were then with caution, since many of the included studies vary widely in their analyzed to verify whether they met the inclusion and exclusion criteria operationalization of aspects of pornography use. Furthermore, the lack developed for the study. For the current study, three specific criteria for of consistency limits the practical application of these findings. Finally, inclusion in the review were established. since the methods for assessing pornography use are relatively het- First, only articles assessing use of pornography were included in erogeneous, less confidence can be placed in the quality of the assess- the review. Articles assessing only perceived addiction, opinions about ments, since they lack the reliability and validity afforded through use pornography, or assessing a partner's use were not included in the in multiple studies. Although the review provided by Short et al. (2012) study. While these are important aspects of research on pornography, was an important examination of the state of pornography use research, the purpose of the current study was to specifically examine how por- it was not without its limitations. The authors limited the studies ana- nography use is operationalized. Additionally, studies were included lyzed in the systematic review to the “PubMed” and “PsycInfo” data- only if they provided information on the items used to assess porno- bases, excluding some studies that may be found only in such databases graphy consumption. If the authors did not give any details about the as “Criminal Justice Abstracts” and “SocIndex.” Additionally, Short items or response options, they were not included in the study. et al. (2012) eliminated studies that examined offender populations. Secondly, only quantitative studies were included in the review. Finally, Short et al. (2012) focused on studies that examined the re- Although qualitative research provides rich information for the study of lationship between pornography use and sexual coercion, rather than pornography consumption, one of the purposes of the current study was including research examining the relationship pornography has with a to identify a more consistent, standardized, and valid way of assessing variety of other behaviors. While this was understandable, given the pornography use, which is not the focus of open-ended qualitative

170 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179

Table 1 Descriptive statistics of the definition of pornography in studies.

N (%)

Provided a definition 58 (18.5) Defined by content 39 (67.3) Content and intended use of material 17 (29.3) Defined by exclusion of material 11 (19.0) Medium-specific 10 (17.2) Intentional use of material 2 (3.4)

3.1. Defining pornography

Among the 313 articles included in the study, less than 20% pro- vided a definition of pornography to respondents (see Table 1). Defi- nitions across these 58 studies varied significantly. The descriptive statistics in Table 1 provide a breakdown of the themes across the provided definitions. The most popular theme identified was defining pornography in terms of specific content. For example, Peter and Valkenberg (2009) defined pornography as “pictures with clearly ex- posed genitals, movies with clearly exposed genitals, pictures with people having sex, and movies with people having sex” (p. 416). Si- Fig. 1. Visualization of gathering and screening process. milarly, in 11 of the studies included in the review, the researchers defined pornography by explicitly excluding certain types of material. approaches. Thirdly, only articles available in English were included in Kraus, Rosenberg, and Tompsett (2015) included a statement in their fi the review. Finally, articles assessing the use of only de nition indicating that any materials containing partially nude or were excluded from the review. After reviewing the abstracts and ap- nude individuals who were not engaged in sexually explicit behavior, plying the inclusion and exclusion criteria, 797 articles remained. The such as underwear catalogues, were not to be considered pornography. fi full-text of each of these articles was then accessed to further assess Træen and Daneback (2013) mirrored this approach in their de nition “ whether the inclusion or exclusion criteria had been met. This detailed of pornography, stating that materials containing men and women fi examination of inclusion and exclusion criteria for each study resulted posing or acting naked (/-girl, advertisements, lm, etc.), and in a drop from 797 articles to 313 articles found to be relevant for the which do not contain clear and explicit sexual acts are not regarded as ” current study. A visual description of the method used in the systematic pornography in this study (p. 42). fi review can be found in Fig. 1. Another theme observed in the de nitions of pornography, was The concepts of interest for the current review include the oper- highlighting the intended use of the material. Although only two of the fi ationalization of pornography use and whether a definition of porno- studies exclusively de ned pornography as material intended for the graphy was provided to the respondents in the study. For oper- sexual arousal of the viewer (Lam & Chan, 2007; Willoughby, Young- ationalization, the review will focus on the three most common ways Peterson, & Leonhardt, 2018), a larger portion of these studies used it in that pornography use is assessed: frequency of use, type of pornography conjunction with the content of the material. For example, beyond fi used, and pornography use habits. A database of the 313 articles in- de ning pornography as including materials depicting various sexual cluded in the review was developed where the authors coded whether behaviors (such as vaginal intercourse or group sex, masturbation), “ fi the article included items assessing the concept, the type of items and Kraus et al. (2015) also stated that pornography is de ned as any response choices provided to respondents, whether the article used an material designed to cause or enhance sexual arousal or excitement in ” existing assessment of pornography use, and whether a definition of the viewer (p. 116). Overall, seventeen of the studies providing a de- fi fi pornography was provided to respondents. This was created in order to nition of pornography de ned it in this manner. fi provide descriptive statistics on the operationalization of pornography A less common theme found among pornography de nitions, was fi use across the studies. limiting the de nition to material accessed on certain mediums, with the majority of these approaches specifying internet as the medium (Beyens, Vandenbosch, & Eggermont, 2015; Braun-Courville & Rojas, 3. Results 2009; Cranney, 2015; Lam & Chan, 2007; Peter & Valkenburg, 2010, 2011, 2012; To, Iu Kan, & Ngai, 2015; Vandenbosch & Oosten, 2018). fi The description of the results from the systematic review will pro- This approach was used in 10 of the studies that provided a de nition. fi gress, first, by discussing studies that used a definition of pornography. Finally, one of the studies provided a statement in the de nition in- fi This will involve describing patterns that emerged in the definitions and tended to normalize the use of pornography. More speci cally, the re- fi “ descriptive statistics on the use of certain types of definitions. Next, the searchers ended their de nition of pornography by stating many ” operationalization of use frequency will be presented. Item stems, people use and enjoy pornography (Goldsmith, Dunkley, Dang, & which contain the question respondents must answer, and item re- Gorzalka, 2017; p. 154). sponse options will be described, in order to provide this information. Following this, the assessment of pornography use habits other than 3.2. Frequency of pornography use frequency will be included. The type of pornography used will be presented through describing the content assessed in the studies, and The most common method of operationalizing pornography use then providing a description of item response types. Finally, the use of across the 313 articles included in this study was through measuring instruments in pornography research will be presented. frequency of pornography consumption, with almost 98% of the studies using this approach (see Table 2). The specific measurement of fre- quency, however, varied widely. Specifically, stems for items varied in regards to the language used to describe pornographic material, the

171 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179

Table 2 pornography use frequency: assessing intentionality of use and using Descriptive statistics for the operationalization of pornography use frequency. pornography for masturbation. For the former, almost 6% of studies fi N (%) speci ed that the items were assessing intentional pornography use. These studies included some sort of indicator in the question signaling Assessed frequency 305 (97.4) to the respondent that accidental use was not to be accounted for. Terminology Finally, in operationalizing frequency of use, some researchers specified Porn, pornography, pornographic material 207 (67.9) Content specific 28 (9.2) that they were assessing only use of pornography to supplement sexual X-rated 23 (7.5) behavior; either alone or in the company of another individual (4.3%). Sexually explicit media/material 21 (6.9) This approach was less common, and was often used in conjunction /cybererotica 16 (5.6) with the assessment of frequency of solitary exposure. Multiple methods 8 (2.6) Adult materials 1 (0.3) Not specified 1 (0.3) 3.2.2. Item response options Modality Although the prior section dealt with variance among item stems for None specified 133 (43.6) frequency of pornography use, pornography assessment researchers Multiple (for example, internet, television, or magazines) 80 (26.2) also varied in their use response options given to respondents (see Internet 62 (20.3) Movies/films 28 (9.2) Table 2). First, the level of measurement used in the response choices Magazines 2 (0.7) will be addressed. In the majority of studies (67.1%), subjects re- Intentional use 17 (5.6) sponded to the item using some sort of Likert-type response. Although Use for sexual activity 13 (4.3) this was the most common response option given to respondents, re- Likert-type response items 210 (67.1) searchers varied widely in the types of anchors used. The most common Monthly, daily, weekly, etc. 149 (71.0) Number of times 27 (12.9) method used in these studies was asking the individual to indicate their Never, sometimes, rarely, etc. 25 (11.9) frequency of use from anchors indicating monthly, weekly, or daily use. Minutes or hours 5 (2.4) For instance, Wright, Sun, and Steffen (2018) used response options fi Unspeci ed 3 (1.4) that ranged from never to almost daily, while Kohut, Balzarini, Fisher, Multiple methods 1 (0.5) Dichotomous items 49 (15.7) and Campbell (2018) used response choices that range from never to Continuous items 29 (9.3) several times a day. This approach was used in over two-thirds of the Number of minutes/h 18 (62.1) studies that used Likert-type items in their study. The second most Number of sessions 7 (24.1) common anchors involved respondents choosing from a provided fi Number of pornographic lms 4 (13.8) number of times (i.e., 0, 1–5, 6–10, 11 or more) an individual used pornography in a set time period, such as within the past 12 months (Grubbs, Stauner, Exline, Pargament, & Lindberg, 2015). Additionally, medium used to view pornography, and the exact nature of the activity researchers also asked respondents to select from this type of anchor to being assessed. indicate frequency of pornography use in a “typical or average week” (Kraus & Rosenberg, 2016; p. 141). This approach was used in about 3.2.1. Item stems 13% of studies. Researchers used a bevy of terms when referring to pornography, While many researchers preferred to use anchors indicating a spe- with the majority (67.9%) using the term “pornography,”“porn,” or cified frequency unit, about 12% of the studies contained anchors with “pornographic material.” Besides this approach, a portion of re- basic Likert-type response choices. For example, anchors provided by searchers described, in the question, the content that they were refer- Tylka (2015) to answer the stem “I view pornography (internet por- ring to (9.2%). For example, Galovan, Drouin, and McDaniel (2018) nography sites, magazines, DVDs, videos, etc.)” were never, sometimes, assessed the frequency an individual viewed content depicting nude often, usually, and always (p. 100). The anchors used by Brand et al. individuals or graphic sexual acts. Other approaches were much more (2011) were specific to the behavior being assessed, with response specific in the content that was described. Morrison, Morrison, and choices ranging from never to always when online. The least common Bradley (2007) assessed the frequency in which individuals viewed anchors used by researchers using Likert-type responses contained “one or more males and/or females engaging in oral, vaginal, and/or choices accounting for number of hours or minutes spent viewing anal sex” (p. 36). Rather than using “pornography,” some researchers pornography in a set period of time (2.4%). Levin, Lillis, and Hayes referred to “sexually explicit media” or “sexually explicit materials” (2012), after asking respondents to indicate average number of hours (6.9%), and some referred to X-rated materials (7.5%), when assessing they used pornography in a typical day, required respondents to select the frequency of pornography consumption. Finally, a smaller portion from response choices ranging from never to more than six hours. of researchers used language such as erotica or cybererotica (5.6%), Bersamin, Bourdeau, Fisher, and Grube (2010) used similar anchors in adult materials (0.3%), or multiple terms (i.e., pornography and sexu- their study, but asked their respondents to indicate average use over a ally explicit material; Blanc, Byers, & Rojas, 2018) in less than 1% of the week. Finally, a study conducted by Tomić, Burić, and Štulhofer (2017) studies. One of the studies did not specify the term used to refer to used two types of anchors to assess frequency of pornography use. pornographic material. Anchors including responses ranging from “never” to “several times a Beyond using different terms to refer to the material, researchers day” were used in response to items assessing frequency of use in the also varied widely in the mediums used to view pornography accounted past six months and two years, and anchors ranging from “zero” to “10 for in their questions. The majority of items did not specify the medium or more times” were used in response to an item assessing number of used to view pornography (43.6%), but when the researchers did ask, times pornography was used in the past week (Tomić et al., 2017). the most used approach was to account for multiple modalities, which Three of the studies included in the review did not provide the response included viewing pictures or images on the internet, in magazines, or options given to respondents. on television (26.2%). For studies that assessed only one specific Simply asking the respondents whether or not they were exposed to modality, the most common was internet pornography, with about 21% pornographic materials was the second-most common approach to as- of studies assessing internet pornography use specifically. Following sessing frequency of use, which was used in about 16% of studies. This internet pornography, the most common modalities accounted for were approach was most often used by researchers examining data from the movies/films (9.2%) and magazines (0.7%). General Social Survey (GSS), which contained one item assessing Two more themes were discovered among item stems for whether or not the individual viewed an x-rated film in the past

172 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179

12 months. Other examples include asking respondents whether or not Table 4 they had viewed a pornographic film in the past month (Luder et al., Descriptive statistics for the operationalization of type of por- 2011), whether they currently use pornographic material (Williams, nography. Cooper, Howell, Yuille, & Paulhus, 2009), or if the individual had ever N (%) intentionally viewed pornography (Albright, 2008). Continuous response items were used most infrequently in the in- Type 61 (19.5) cluded studies (9.3%). Researchers using this approach required re- Risky sexual behaviors 41 (67.2) Physical aggression 31 (50.8) spondents to respond numerically to an open-ended question. Over half Mainstream pornography 29 (47.5) of the studies containing continuous response items required the re- Humiliation 26 (42.6) spondent to indicate a specific number of hours or minutes they used Paraphillia 26 (42.6) pornographic material (62.1%). For example, Laier, Pekal, and Brand Consent 15 (24.6) Child 15 (24.6) (2014) asked respondents to indicate average number of minutes in a Item response type month spent viewing pornography. Some researchers, such as Harper Dichotomous 28 (45.9) and Hodgins (2016), required respondents to provide the specific Likert-type 21 (34.4) number of pornography use sessions per month. This approach was Continuous 5 (8.2) used by about a quarter of the studies using this method. Alternatively, Multiple types 4 (6.6) ff Not provided 3 (4.9) Rasmussen and Bierman (2016) used a di erent approach altogether, Used type scales 12 (19.7) asking their respondents to indicate the number of pornographic films Multiple scales 7 (58.3) they watched in the previous year. Asking respondents to indicate Violent 2 (16.7) number of films was the least common approach, being used in about Non-mainstream 1 (8.3) Gonzo 1 (8.3) 14% of studies assessing frequency in this manner. Male-female centric 1 (8.3)

3.3. Habits (50.8%). Among these studies, researchers referred to this type of ma- Nearly 30% of the studies included in the review assessed habits terial simply as “violent,”“sexually violent,” or “” pornography. In related to an individual's pornography use (see Table 3). Among habits, a small number of studies, researchers asked respondents to indicate researchers examined the typical modality used to view pornography whether the actors in the porn they view ever appeared to be hurt. (48.3%), subject's age at first exposure to pornography (35.6%), typical Additionally, some researchers distinguished between “consensual” duration of use (29.9%), motivation for use (20.7%), who the in- violence and non-consensual violence in their assessment of physical dividual uses pornography with (17.2%), monetary investment aggression. Finally, a study conducted by Foubert and Bridges (2017a, (10.3%), and typical location of use (6.9%). 2017b) used data from a scale containing 44 items pertaining to specific violent behaviors, though their scale was designed to assess violent and 3.4. Type of pornography used degrading pornography, not just violent pornography. Researchers also assessed non-violent or mainstream forms of por- fi The third most common way to operationalize pornography use was nography, such as heterosexual sex, or pornography speci c to actor asking participants what type of pornography they viewed. Assessing characteristics, such as breast size or age. These categories/acts were type of pornography viewed was used in about 20% of the studies in- assessed in just less than half of the studies examining type. Similarly, cluded in the review (see Table 4). However, methods used to assess researchers also examined pornography containing humiliation in type of pornography viewed varied widely. about 43% of the studies. Humiliation most commonly assessed by asking respondents if they specifically viewed bondage, discipline, sadism, and masochism (BDSM) pornography. Other common methods 3.4.1. Content involved identifying respondents who watched pornography where a Risky sexual behavior was the most common type of pornography male actor ejaculates on another actor's face, forced gagging during oral respondents were asked about, with nearly 70% of the studies mea- sex, an actor urinating or defecating on another actor, and name-calling suring type asking respondents to report their use of pornography de- or slurs. Finally, other researchers specifically asked if the individual picting “risky sexual behavior.” Researchers using this method typically used “degrading” pornography, and similar to their operationalization defined risky sexual behavior through anal intercourse, and more spe- of type, Foubert and Bridges (2017a, 2017b) used a scale assessing cifically, anal intercourse. “degrading” behaviors using items referring to specific acts, such as without any prophylactics. Other researchers defined risky sexual forced gagging and ejaculation on an actor's face. behavior in pornography as containing group sex, general condom Pornography containing paraphilia was also assessed in about 43% usage, or scenes depicting “hook-ups” between actors. of studies. These studies measured a wide variety of content, ranging Physical aggression was the second most common type of porno- from fetishes involving leather, torture, and “kink”. Other studies in- graphy assessed among studies examining type of pornography used cluded items asking the respondent if they viewed bestiality porno- graphy. Finally, Tripodi et al. (2015) assessed a lengthy list of para- Table 3 philic content, ranging from necrophilia to pedophilia. Descriptive statistics for the operationalization of porno- Consent was assessed in a quarter of the studies examining type of graphy use habits. pornography. This was evaluated in many studies by asking the re- N (%) spondents if they specifically viewed pornography containing acts of rape. Other researchers described the situation of the scene more vi- Assessed habits 87 (27.8) Typical modality 42 (48.3) vidly in their items, asking respondents if they viewed pornography Age at first exposure 31 (35.6) where the woman was coerced or expressed a specific lack of consent, Duration 26 (29.9) or was unable to give consent because she was drugged, intoxicated, or Motivation 18 (20.7) asleep. About a quarter of studies also assessed whether the respondent Solitary/group use 15 (17.2) Monetary investment 9 (10.3) viewed child pornography. The majority of these studies simply asked Typical location of use 6 (6.9) whether the individual viewed pornography containing sex with

173 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179 children, but did not inquire about the specific age-ranges. In only one 2014). The OPS (Seigfried, 2007), used by three of the studies in the of the included studies, the researchers asked if the respondent had review (Seigfried-Spellar, 2014, 2016; Seigfried-Spellar & Rogers, viewed pedophilic pornography (Tripodi et al., 2015). It should also be 2013), did not have any support for the reliability or validity of the noted, that Tripodi et al. (2015) did not provide a specificdefinition of instrument provided, either in previous studies or in the study in which pedophilic pornography. it was used. For the SEMQ (Wryobeck & Wiederman, 1999), the only evidence of 3.4.2. Item response options reliability or validity provided was demonstrated through providing Researchers assessing the type of pornography an individual con- Cronbach's alpha for the study in which it was used (Wurtele, Simons, & sumed typically used dichotomous (45.9%) or Likert-type (34.4%) item Moreno, 2014). The provision of Cronbach's alpha for evidence of re- response options. In a small portion of the studies assessing type, con- liability in the study of use was the sole method of providing evidence tinuous items (8.2%) and questions using multiple item response for reliability and validity in the remainder of studies using instruments choices (6.6%) were used. About 5% of the studies did not provide any (Foubert & Bridges, 2017a, 2017b; Shaughnessy, Byers, Clowater, & information on the response choices provided to their subjects. About Kalinowski, 2014; Zheng et al., 2017). 20% of the studies assessing type used multiple items comprising scales, but these scales varied widely in their scope and purpose. The most 4. Discussion common type of scale created assessed use of violent pornography (Baer, Kohut, & Fisher, 2015; Davis, Carrotte, Hellard, & Lim, 2018). As pornography use becomes increasingly an increasingly universal One study created scales measuring male-centric and female-centric behavior, the need for accurate and reliable research on the potential pornographic material (French & Hamilton, 2018), whereas the scales effects of pornography grows more salient. This is especially true for created by Wright et al. (2015) specifically measured interest in researchers examining some of the more problematic relationships “Gonzo” pornography, an extremely aggressive and graphic . uncovered by research, such as pornography use and sexual coercion. Cranney and Stulhofer (2017) created a scale measuring use of non- While a wide body of research has established that there is a relation- mainstream pornography, such as BDSM. Finally, Brand et al. (2011) ship between pornography use and sexually coercive behavior, very created a scale assessing the frequency of all types of pornography. little has been done to move beyond this correlational understanding. Research attempting to extend this line of inquiry further, by in- 3.5. Instrument use corporating multiple aspects of pornography use that show promise, is the exception rather than the rule. Furthermore, inconsistencies in the Finally, studies included in the review were examined to identify conceptualization and operationalization of pornography use call into the use of validated and reliable measures. Results indicated that only question the accuracy and reliability of these findings, while also sti- less than 5% of the studies used existing instruments to assess porno- fling the field from developing a standardized measure of pornography graphy use (see Table 5). The measures included assessed several dif- use. The purpose of the current study was to conduct a systematic re- ferent aspects of pornography consumption, with some designed as a view of research assessing pornography use published in peer-reviewed general or comprehensive assessment (60%), while others specifically journals over the last decade, to gain an understanding of how promi- examined frequency of use (26.7%) or frequency of use and type nent these issues are in the field, and to use these findings to provide (13.3%). For comprehensive or general assessments of pornography future direction in the pornography use field of research. use, the most common measure was the Pornography Consumption Questionnaire (PCQ; Hald, 2006). Researchers also used the Online 4.1. Defining pornography Pornography Survey (OPS; Seigfried, 2007) as a general pornography use assessment. For measures assessing frequency of use, the Sexually Broadly, the findings of the current study indicate similar metho- Explicit Media Questionnaire (SEMQ; Wryobeck & Wiederman, 1999) dological issues in pornography research as reported by Short et al. was used. Finally, the Exposure to Sexual Materials Scale (ESMS; Frable, (2012): the conceptualization of pornography (and its use) in peer-re- Johnson, & Kellman, 1997) was used to assess both frequency of use viewed research is inconsistent. Similar to Short et al. (2012), the re- and type of pornography used. sults of the current study showed that less than 20% of the studies As for psychometric validation, the instruments lacked the extensive provided a definition of pornography use to respondents. Significant validation required to establish the measure as a reliable and valid variation in definitions were also identified, and some of these defini- measure of pornography use. For the most commonly used instrument, tions present substantive issues, such as causing respondents to conflate the PCQ (Hald, 2006), limited evidence for validity and reliability was other media with pornographic material, or conflicting with other ex- provided. For instance, Hald and Malamuth (2008) provided some isting definitions of pornography. For instance, Peter and Valkenberg evidence for the reliability of sections of the PCQ in their study. This (2009) defined pornography as material containing depictions of was completed by providing results of Cronbach's alpha and factor clearly exposed genitals and sexual acts. Although satisfying a content- analyses. Hald, Malamuth, and Lange (2013) also provided evidence for related aspect of pornography use, other types of media, such as movies the reliability of the instrument using similar analyses. The majority of or cable TV shows that contain sexually explicit material, may be the studies using the PCQ, however, did not provide any psychometric considered pornography based on this definition. Additionally, some assessment of the instrument in their study (Hald & Malamuth, 2015; researchers specify that magazines, such as Playboy, should not be Hald & Mulya, 2013; Hesse & Pedersen, 2017; Wijaya Mulya & Hald, considered pornography (Kraus et al., 2015; Træen & Daneback, 2013). The exclusion of such materials as “pornographic” conflicts with the Table 5 conceptualization of other researchers who do consider this type of Descriptive statistics for validated instruments used in pornography material pornographic (González-Ortega & Orgaz-Baz, 2013; Kohut use research. et al., 2018; Kvalem, Træen, & Iantaffi, 2016; Rasmussen, Rhodes, Ortiz, & White, 2016; Wetterneck, Burgess, Short, Smith, & Cervantes, N (%) 2012; Wright & Tokunaga, 2016). Used instrument 15 (4.8) While the majority of studies used a content-specific approach to Purpose of instrument defining pornography, just under a third of the studies defined porno- General/comprehensive measure 9 (60.0) graphy by the content and the intended use of the material. That is, the Frequency of use 4 (26.7) fi Frequency of use and type used 2 (13.3) researchers speci ed that the material was created with the intent of enhancing the sexual activity of the viewer. This approach is beneficial

174 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179 for two reasons. First, by adding a second caveat to the definition, the pornography use have noted that, while “most-recent event” dichot- intended use of the material, this may prevent respondents from con- omous items are desirable for assessing prevalence of use, this is not flating non-pornographic media containing sexually explicit material recommended for the assessment of frequency (Regnerus et al., 2016). with pornography. Additionally, research has shown that there is sup- This means that data sets gathered for examining the prevalence of port for the notion that pornography is a multidimensional construct pornography use, such as the pornography item in the GSS, are not (Busby, Chiu, Olsen, & Willoughby, 2017). Thus, applying a two-part good proxies of pornography use frequency. Furthermore, research has definition to pornography is not only appealing at face value, but is found that meaningful frequency thresholds beyond which porno- supported in research on the conceptualization of pornography. Re- graphy use becomes predictive of sexual coercion exist (Marshall et al., searchers have highlighted the high level of variance in the way in- 2017). In conclusion, it seems that measures of pornography use lacking dividuals perceive media to be pornographic or not (Willoughby & variance may limit the statistical strength necessary to accurately Busby, 2016), and because of this, researchers assessing pornography identify the nature of the relationship between pornography use and should provide quality definitions of pornography to increase the ac- sexually coercive behaviors. curacy and reliability of their findings. Providing quality definitions of Together, these findings provide two implications for the assessment pornography would also give researchers the ability to compare find- of frequency of use moving forward. First, researchers should avoid ings across studies. Simply adding quality definitions of pornography using dichotomous items. Secondary data sets may preclude re- would advance this research area significantly, since less than 20% of searchers' ability to use such methodologies, but this practice should be the studies included in the review used any definition at all. Further- avoided if possible, especially in light of research indicating that more, from the review findings provided in this study, pornography meaningful thresholds beyond which certain levels of usage elevate the definitions that specify content and intended purpose would be the relationship between pornography use and sexual coercion. Finally, most advantageous moving forward. researchers should be conscientious of the terminology they use when writing surveys. Studies have shown that use of antiquated language, 4.2. Operationalization of pornography use such as X-rated material or erotica, creates discrepancies in the way respondents answer survey items (Regnerus et al., 2016). The over- Discrepancies also existed in the operationalization of pornography whelming majority of studies refer to pornography as simply porno- use. These issues centered around the narrow focus of use frequency for graphy, porn, or pornographic material, and because of this, re- the overwhelming majority of studies, the inconsistent terminology in searchers should continue using such terminology to establish a sense of the use items, and the near universal practice of using researcher-gen- consistency. As an alternative, however, some researchers have begun erated items to assess pornography use rather than a standardized employing the terms “sexually explicit media” or “sexually explicit measure. material.” Use of these terms may be beneficial in order to combat any taboo or self-consciousness terms like “porn” may conjure in re- 4.2.1. Frequency of use spondents. Future research should compare response rates in surveys The majority of studies operationalized pornography use by use employing both methods, in order to examine which approach may be frequency. In particular, descriptive statistics indicated that frequency more efficacious. was assessed in nearly 98% of the studies included in this review. The body of research on pornography use does seem to indicate some 4.2.2. Habits consistency in this regard, but the methodologies through which fre- The second most common aspect of pornography use assessed by quency is addressed widely vary. First, this review found that sig- researchers were habitual factors other than frequency of use, which nificant variance in terminology used to refer to “pornography.” Among were assessed in just under 30% of the studies. These factors included frequency items, two-thirds of the studies simply referred to porno- the typical modality of use, age at first exposure, duration, motivation, graphy, porn, or pornographic materials, while the remaining one-third solitary or group use, monetary investment, and typical location of use. was heterogeneous in the terminology used. These studies used alter- Compared to frequency of use, these other habits were assessed con- native terms such as erotica (Maddox, Rhoades, & Markman, 2011; siderably less. Researchers have indicated that frequency of use is an Minarcik, Wetterneck, & Short, 2016; Muusses, Kerkhof, & Finkenauer, important aspect to assess, especially in regards to sexually coercive 2015), adult materials (Rechter & Sverdlik, 2016), sexually explicit behaviors (Wright et al., 2016), but studies have also shown that other media (Braun-Courville & Rojas, 2009; Nelson, Pantalone, Gamarel, & habitual factors matter as well. For instance, studies have shown that Simoni, 2016; To et al., 2015; Traen et al., 2014, Træen et al., 2015), age at first exposure (Burton et al., 2010; Mancini et al., 2012) and and X-rated materials (Kohut, Baer, & Watts, 2016; Patterson & Price, number of modalities used to view pornography (Marshall et al., 2017, 2012; Wright, Bae, & Funk, 2013). Although these diff erences may 2018) are significantly related to sexual coercion. Confidence in these seem trivial, researchers have highlighted how terms, such as erotica findings, however, is limited due to the small amount of studies as- and adult materials, may conjure an image of substantively different sessing these factors. Less than 10% of studies assessed age at first ex- material compared to pornography (Willoughby & Busby, 2016). Some posure, and for modalities, only two studies assessed the number of of these terms (i.e. X-rated materials) are also dated, according to some modalities used by an individual. Movement towards a more complete researchers, which may result in an underestimation of pornography understanding of the effects of pornography use will not be possible use (Regnerus, Gordon, & Joseph, 2016). This indicates that findings for without research assessing many variables, and the results of this re- studies using varying terms may either be capturing use of different view indicate that the majority of studies have not accomplished this. materials or underestimating the prevalence and frequency of porno- Certain habitual factors do show some promise, specifically in regards graphy use. to the relationship between pornography use and sexual coercion, so In addition to the inconsistent terminology, results indicated there researchers should continue to include items assessing these factors, in were also a large amount of studies that assessed frequency of use order to provide a more complete and consistent operationalization of through a single dichotomous item. Many of these studies used sec- pornography use. ondary data from the GSS, which measured pornography use with a single item asking whether an individual viewed a pornographic film in 4.2.3. Type of pornography used the past 12 months. A small portion of studies using a dichotomous Researchers assessed the type of pornography viewed by an in- item, did not use secondary data, however, and intentionally measured dividual in less than 20% of the studies included in the review. This was pornography use with a single dichotomous item. This is problematic the least common of the approaches, when compared to frequency and for multiple reasons. Researchers examining the measurement of other habits. Much like the previous discussion regarding habitual

175 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179 factors other than frequency of use, this lack of assessment in the ma- finding, several caveats exist. First, studies using existing measures jority of studies could prove to be problematic. In fact, a larger body of accounted for less than 5% of the studies included in this review. This literature exists stressing the importance of the kind of pornography an indicates that the overwhelming majority of studies are still conducted individual uses (Foubert, Brosi, & Bannon, 2011; Romito & Beltramini, using researcher-generated items. Additional problems involve the in- 2011; Wright et al., 2016; Ybarra, Mitchell, Hamburger, Diener-West, & struments used in these studies. The four instruments used by re- Leaf, 2011). Some researchers have also indicated that type of porno- searchers were developed between 1996 and 2007, indicating that the graphy may matter more than frequency of pornography use (Kingston most recent instrument is over a decade old. Technological advance- et al., 2008). This finding, however, must be interpreted with caution, ments over the previous decade, such as internet speed, the introduc- as type is assessed in a significantly smaller amount of studies. What is tion of smartphones, and the prevalence of providing the evident, however, is that researchers interested in understanding the ability to stream videos in web browsers (otherwise called “tube” sites), effects of pornography consumption must also examine what an in- provide copious amounts of free pornographic material of all kinds dividual is looking at when they consume pornography, and how the (Buhi, Daley, Fuhrmann, & Smith, 2009; Livingstone & Smith, 2014; content of the pornography may interact with other habitual factors to Regnerus et al., 2016), calling into question the ability of dated mea- influence behavior. sures to accurately assess pornography use, frequency, and type. Discrepancies also existed in how researchers operationalized cer- Furthermore, none of the instruments included in the studies un- tain types of pornography. Of particular concern to the current study is derwent the stringent psychometric assessment necessary to establish the operationalization of violent pornography. First, violent porno- these instruments as reliable and valid. For instance, the only instru- graphy use was operationalized inconsistently across studies. The ma- ment whose psychometric assessment went beyond providing jority of studies assessed whether or not an individual reported using Cronbach's alphas was the PCQ, which underwent a factor analysis in violent, rape, or sexually violent pornography (Baer et al., 2015; one study (Hald & Malamuth, 2008). The use of instruments is only as Burton, Duty, & Leibowitz, 2011; González-Ortega & Orgaz-Baz, 2013; beneficial as the psychometric integrity of the instrument, and this can Hald & Štulhofer, 2016; Mattebo, Tydén, Häggström-Nordin, Nilsson, & be established only through repeated examinations of validity and re- Larsson, 2016; Romito & Beltramini, 2015; Seto et al., 2015; Short, liability. Thus, in order to increase the use of instruments in studies of Kasper, & Wetterneck, 2015; Štulhofer, Buško, & Landripet, 2010; pornography use, researchers should focus on either developing psy- Wijaya Mulya & Hald, 2014), while some researchers asked individuals chometrically sound instruments, or validating previously developed to report whether the actors in the pornography they watched were instruments. It is possible that the pervasive use of researcher-generated hurt or in pain (Bridges & Morokoff, 2011; Romito & Beltramini, 2011; pornography use items also found in Short et al. (2012) study may not Ybarra et al., 2011). Finally, in one of the studies, the researchers used be because of a choice of the researcher, but rather, the lack of valid an assessment containing a multitude of different aggressive behaviors, and reliable pornography use instruments. to assess the level of aggression within the pornography (Foubert & Bridges, 2017a, 2017b). These methods, though purporting to assess 4.3. Limitations the same concept, may vary widely in what they are actually mea- suring. The current study built upon the research of Short et al. (2012). Second, some of the methodologies used may be problematic. The That is not to say that the current study is without limitations. First, majority of studies assessed consumption of violent pornography by while the current study used broader search terms, not all potential simply asking individuals whether or not they viewed violent, sexually search terms were employed (i.e., erotica, X-rated material, adult ma- violent, or . Though this may seem to be the most terial). While these terms may arguably represent material that is obvious approach to assessing use of violent pornography, researchers substantively different from “pornography,” the confl ation of these have indicated that even pornography labeled “mainstream” can con- terms was made apparent by the results of this study. Future studies tain acts of physical aggression (Bridges et al., 2010). This indicates that may expand upon this methodology by including all relevant termi- items assessing whether an individual watches “violent” or “rape” nology. Furthermore, not all potential databases were used in the cur- pornography may fail to capture these individuals, since they perceive rent study. This means that some papers not included in PsycInfo, So- the pornography they are viewing as mainstream. In contrast to this cindex, or Criminal Justice Abstracts would be excluded. In addition to approach, Foubert and Bridges (2017a, 2017b) used a measure con- terms and databases, the current review only included studies published taining 44 items assessing the involvement of various acts of aggression, in English. Finally, only quantitative studies were included in the re- such as choking, slapping, or punching. Although use of this approach view. This does not speak to the value qualitative work contributes to has been limited, findings from content analyses on pornography use research on pornography use. The purpose of the current study was to suggest that if researchers are interested in assessing the type of por- examine the way researchers have conceptualized pornography and nography used by an individual, the best method to use moving forward operationalized its use, limiting the contribution qualitative work would involve behavior-specific items, rather than asking individuals would make to the current study. what genre of pornography they regularly view (Bridges et al., 2010). 5. Conclusion 4.2.4. Instrument use Finally, the results of this review revealed that the overwhelming Pornography use has become very common in society. With re- majority of studies used researcher-generated items to assess frequency searchers indicating that more individuals than ever are consuming of pornography use, other habits of use, and the type of pornography pornographic material, the need for research examining the potential consumed, which partially mirrors the findings of the review by Short impacts of pornography is more evident than ever. This is especially et al. (2012). The inconsistencies in the operationalization of use fre- true for some of the possible harmful outcomes of pornography use, quency and type of pornography used, noted in the previous sections, such as engaging in sexually aggressive behaviors. Researchers have, in are a direct result of the use of researcher-generated questions rather part, answered the call and provided ample evidence that pornography than a standardized assessment measure. This is especially problematic, use and sexual coercion are in fact related, but until the field moves because it precludes researchers from comparing results between stu- beyond a correlative understanding of this relationship, practical im- dies with varying methodologies. It should be noted, however, that plications are limited. Additionally, inconsistent methodologies limit contrary to the review by Short et al. (2012), the current study did the ability of researchers to synthesize these studies into meta-analyses. reveal that some studies were conducted using existing instruments The purpose of the current study was to assess the conceptualization assessing pornography consumption. While this was an encouraging and operationalization of pornography definitions and pornography use

176 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179 in studies conducted over the past decade, in order to provide an young heterosexual Australians see in pornography? A cross-sectional study. Journal overview of the various ways researchers are defining and measuring of Sex Research, 55(3), 310–319. https://doi.org/10.1080/00224499.2017.1417350. Foubert, J. D., & Bridges, A. J. (2017a). Predicting bystander efficacy and willingness to pornography use. Although the results indicated that pornography use intervene in college men and women: The role of exposure to varying levels of vio- research is extremely heterogeneous and largely ignores important as- lence in pornography. , 23(6), 692–706. https://doi.org/10. pects of pornography use that may help the field move beyond a simple 1177/1077801216648793. Foubert, J. D., & Bridges, A. J. (2017b). What is the attraction? Pornography use motives correlative understanding of the relationship between pornography use in relation to bystander intervention. Journal of Interpersonal Violence, 32(20), and sexual coercion, some encouraging findings were revealed. Moving 3071–3089. https://doi.org/10.1177/0886260515596538. forward, research will benefit from defining pornography in standar- Foubert, J. D., Brosi, M. W., & Bannon, R. S. (2011). Pornography viewing among fra- ff dized ways, adopting methodologies assessing multiple aspects of por- ternity men: E ects on bystander intervention, acceptance and behavioral intent to commit . & Compulsivity, 18(4), 212–231. nography use, and establishing reliable and valid assessments of por- https://doi.org/10.1080/10720162.2011.625552. nography use factors. Frable, D. E., Johnson, A. E., & Kellman, H. (1997). Seeing masculine men, sexy women, and gender differences: Exposure to pornography and cognitive constructions of – 1 gender. Journal of Personality, 65(2), 311 355. References French, I. M., & Hamilton, L. D. (2018). Male-centric and female-centric pornography consumption: Relationship with sex life and attitudes in young adults. Journal of Sex – Albright, J. M. (2008). Sex in America online: An exploration of sex, marital status, and & Marital Therapy, 44(1), 73 86. https://doi.org/10.1080/0092623X.2017.1321596. fi sexual identity in internet sex seeking and its impacts. Journal of Sex Research, 45(2), Galovan, A. M., Drouin, M., & McDaniel, B. T. (2018). pro les in the United States 175–186. https://doi.org/10.1080/00224490801987481. and Canada: Implications for individual and relationship well-being. Computers in – Allen, M., D’Alessio, D., & Brezgel, K. (1995a). A meta-analysis summarizing the effects of Human Behavior, 79,19 29. https://doi.org/10.1016/j.chb.2017.10.017. pornography II: Aggression after exposure. Human Communication Research, 22, Goldsmith, K., Dunkley, C. R., Dang, S. S., & Gorzalka, B. B. (2017). Pornography con- 258–283. sumption and its association with sexual concerns and expectations among young – Allen, M., Emmers, T., Gebhardt, L., & Giery, M. A. (1995b). Exposure to pornography men and women. Canadian Journal of , 26(2), 151 162. https://doi. and acceptance of rape myths. Journal of Communication, 45(1), 5–26. org/10.3138/cjhs.262-a2. ’ Baer, J. L., Kohut, T., & Fisher, W. A. (2015). Is pornography use associated with anti- González-Ortega, E., & Orgaz-Baz, B. (2013). Minors exposure to online pornography: ff – woman sexual aggression? Re-examining the confluence model with third variable Prevalence, motivations, contents and e ects. Anales de Psicología, 29(2), 319 327. considerations. Canadian Journal of Human Sexuality, 24(2), 160–173. https://doi. Grubbs, J. B., Stauner, N., Exline, J. J., Pargament, K. I., & Lindberg, M. J. (2015). org/10.3138/cjhs.242-A6. Perceived addiction to internet pornography and psychological distress: Examining Bersamin, M. M., Bourdeau, B., Fisher, D. A., & Grube, J. W. (2010). Television use, sexual relationships concurrently and over time. Psychology of Addictive Behaviors, 29(4), – behavior, and relationship status at last oral sex and vaginal intercourse. Sexuality 1056 1067. https://doi.org/10.1037/adb0000114. ff and Culture, 14(2), 157–168. https://doi.org/10.1007/s12119-010-9066-x. Hald, G. M. (2006). Gender di erences in pornography consumption among young het- – Beyens, I., Vandenbosch, L., & Eggermont, S. (2015). Early adolescent boys’ exposure to erosexual Danish adults. Archives of Sexual Behavior, 35, 577 585. ff internet pornography: Relationships to pubertal timing, sensation seeking, and aca- Hald, G. M., & Malamuth, N. M. (2008). Self-perceived e ects of pornography con- – demic performance. The Journal of Early Adolescence, 35(8), 1045–1068. https://doi. sumption. Archives of Sexual Behavior, 37(4), 614 625. https://doi.org/10.1007/ org/10.1177/0272431614548069. s10508-007-9212-1. ff Blanc, A., Byers, E. S., & Rojas, A. J. (2018). Evidence for the validity of the Attitudes Hald, G. M., & Malamuth, N. N. (2015). Experimental e ects of exposure to pornography: ff ff Toward Sexual Behaviours Scale (ASBS) with Canadian young people. Canadian The moderating e ect of personality and mediating e ect of sexual arousal. Archives – Journal of Human Sexuality, 27(1), 1–11. https://doi.org/10.3138/cjhs.2017-0024. of Sexual Behavior, 44(1), 99 109. https://doi.org/10.1007/s10508-014-0291-5. Brand, M., Laier, C., Pawlikowski, M., Schächtle, U., Schöler, T., & Altstötter-Gleich, C. Hald, G. M., Malamuth, N. N., & Lange, T. (2013). Pornography and sexist attitudes – (2011). Watching pornographic pictures on the internet: Role of sexual arousal rat- among heterosexuals. Journal of Communication, 63(4), 638 660. https://doi.org/10. ings and psychological–psychiatric symptoms for using internet sex sites excessively. 1111/jcom.12037. Cyberpsychology, Behavior and Social Networking, 14(6), 371–377. https://doi.org/10. Hald, G. M., & Mulya, T. W. (2013). Pornography consumption and non-marital sexual 1089/cyber.2010.0222. behaviour in a sample of young Indonesian university students. Culture, Health & – Braun-Courville, D. K., & Rojas, M. (2009). Exposure to sexually explicit web sites and Sexuality, 15(8), 981 996. https://doi.org/10.1080/13691058.2013.802013. Š “ adolescent sexual attitudes and behaviors. Journal of Adolescent Health, 45(2), Hald, G. M., & tulhofer, A. (2016). What types of pornography do people use and do 156–162. https://doi.org/10.1016/j.jadohealth.2008.12.004. they cluster? Assessing types and categories of pornography consumption in a large- ” Bridges, A., Wosnitzer, R., Scharrer, E., Sun, C., & Liberman, R. (2010). Aggression and scale online sample : Corrigendum. Journal of Sex Research, 53(7), 894. https://doi. sexual behavior in best-selling pornography videos: A content analysis update. org/10.1080/00224499.2016.1140487. Violence Against Women, 16, 1065–1085. Harper, C., & Hodgins, D. C. (2016). Examining correlates of problematic internet por- Bridges, A. J., & Morokoff, P. J. (2011). Sexual media use and relational satisfaction in nography use among university students. Journal of Behavioral Addictions, 5(2), – heterosexual couples. Personal Relationships, 18(4), 562–585. https://doi.org/10. 179 191. https://doi.org/10.1556/2006.5.2016.022. 1111/j.1475-6811.2010.01328.x. Hesse, C., & Pedersen, C. L. (2017). Porn sex versus real sex: How sexually explicit ma- Brown, J. D., & L’Engle, K. L. (2009). X-rated: Sexual attitudes and behaviors associated terial shapes our understanding of sexual anatomy, physiology, and behaviour. – with U.S. early adolescents’ exposure to sexually explicit media. Communication Sexuality & Culture: An Interdisciplinary Quarterly, 21(3), 754 775. https://doi.org/10. Research, 36, 129–151. 1007/s12119-017-9413-2. ff Buhi, E. R., Daley, E. M., Fuhrmann, H. J., & Smith, S. A. (2009). An observational study Kingston, D. A., Fedoro , P., Firestone, P., Curry, S., & Bradford, J. M. (2008). of how young people search for online sexual health information. Journal of American Pornography use and sexual aggression: The impact of frequency and type of por- ff College Health, 58, 101–111. nography use on recidivism among sexual o enders. Aggressive Behavior, 34(4), – Burton, D. L., Duty, K. J., & Leibowitz, G. S. (2011). Differences between sexually victi- 341 351. https://doi.org/10.1002/ab.20250. “ mized and nonsexually victimized male adolescent sexual abusers: Developmental Kohut, T., Baer, J. L., & Watts, B. (2016). Is pornography really about making hate to ” antecedents and behavioral comparisons. Journal of Child : Research, women? pornography users hold more gender egalitarian attitudes than nonusers in – Treatment, & Program Innovations for Victims, Survivors, & Offenders, 20(1), 77–93. a representative American sample. Journal of Sex Research, 53(1), 1 11. https://doi. https://doi.org/10.1080/10538712.2011.541010. org/10.1080/00224499.2015.1023427. ’ Burton, D. L., Leibowitz, G. S., & Howard, A. (2010). Comparison by crime type of ju- Kohut, T., Balzarini, R. N., Fisher, W. A., & Campbell, L. (2018). Pornography s associa- venile delinquents on pornography exposure: The absence of relationships between tions with open sexual communication and relationship closeness vary as a function exposure to pornography and sexual offense characteristics. Journal of Forensic of dyadic patterns of pornography use within heterosexual relationships. Journal of – Nursing, 3(3), 121–129. Social and Personal Relationships, 35(4), 655 676. https://doi.org/10.1177/ Busby, D. M., Chiu, H., Olsen, J. A., & Willoughby, B. J. (2017). Evaluating the di- 0265407517743096. ’ mensionality of pornography. Archives of Sexual Behavior. https://doi.org/10.1007/ Kraus, S. W., & Rosenberg, H. (2016). Lights, camera, condoms! Assessing college men s s10508-017-0983-8 Advance online publication. attitudes toward condom use in pornography. Journal of American College Health, – Cranney, S. (2015). Internet pornography use and sexual body image in a Dutch sample. 64(2), 139 146. https://doi.org/10.1080/07448481.2015.1085054. ffi International Journal of Sexual Health, 27(3), 316–323. https://doi.org/10.1080/ Kraus, S. W., Rosenberg, H., & Tompsett, C. J. (2015). Assessment of self-e cacy to 19317611.2014.999967. employ self-initiated pornography use-reduction strategies. Addictive Behaviors, 40, – Cranney, S., & Stulhofer, A. (2017). 'Whosoever looketh on a person to lust after them': 115 118. ffi Religiosity, the use of mainstream and nonmainstream sexually explicit material, and Kvalem, I. L., Træen, B., & Ianta , A. (2016). Internet pornography use, body ideals, and sexual satisfaction in heterosexual men and women. Journal of Sex Research, 54(6), sexual self-esteem in Norwegian gay and bisexual men. Journal of Homosexuality, – 694–705. https://doi.org/10.1080/00224499.2016.1216068. 63(4), 522 540. https://doi.org/10.1080/00918369.2015.1083782. Davis, A. C., Carrotte, E. R., Hellard, M. E., & Lim, M. S. C. (2018). What behaviors do Laier, C., Pekal, J., & Brand, M. (2014). addiction in heterosexual female users of internet pornography can be explained by gratification hypothesis. Cyberpsychology, Behavior and Social Networking, 17(8), 505–511. https://doi.org/10. 1089/cyber.2013.0396. 1 The current reference section only includes articles cited in text. An ex- Lam, C. B., & Chan, D. K. (2007). The use of cyberpornography by young men in Hong haustive reference section of all 313 articles included in the review can be Kong: Some psychosocial correlates. Archives of Sexual Behavior, 36(4), 588–598. provided by the author(s) upon request. Levin, M. E., Lillis, J., & Hayes, S. C. (2012). When is online pornography viewing

177 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179

problematic among college males? Examining the moderating role of experiential Seigfried-Spellar, K. C. (2016). Deviant pornography use: The role of early-onset adult avoidance. Sexual Addiction & Compulsivity, 19(3), 168–180. pornography use and individual differences. International Journal of Cyber Behavior, Livingstone, S., & Smith, P. K. (2014). Annual research review: Harms experienced by Psychology and Learning, 6(3), 34–47. https://doi.org/10.4018/IJCBPL.2016070103. child users of online and mobile technologies: The nature, prevalence and manage- Seigfried-Spellar, K. C., & Rogers, M. K. (2013). Does deviant pornography use follow a ment of sexual and aggressive risks in the digital age. The Journal of Child Psychology Guttman-like progression? Computers in Human Behavior, 29(5), 1997–2003. https:// and Psychiatry, 55(6), 635–654. https://doi.org/10.1111/jcpp.12197. doi.org/10.1016/j.chb.2013.04.018. Luder, M. T., Pittet, I., Berchtold, A., Akre, C., Michaud, P. A., & Suris, J. C. (2011). Seto, M. C., Hermann, C. A., Kjellgren, C., Priebe, G., Svedin, C. G., & Långström, N. Associations between online pornography and sexual behavior among adolescents: (2015). Viewing child pornography: Prevalence and correlates in a representative Myth or reality? Archives of Sexual Behavior, 40(5), 1027–1035. community sample of young Swedish men. Archives of Sexual Behavior, 44(1), 67–79. Maddox, A. M., Rhoades, G. K., & Markman, H. J. (2011). Viewing sexually-explicit https://doi.org/10.1007/s10508-013-0244-4. materials alone or together: Associations with relationship quality. Archives of Sexual Shaughnessy, K., Byers, E. S., Clowater, S. L., & Kalinowski, A. (2014). Self-appraisals of Behavior, 40(2), 441–448. https://doi.org/10.1007/s10508-009-9585-4. arousal-oriented online sexual activities in university and community samples. Mancini, C., Reckdenwald, A., & Beauregard, E. (2012). Pornographic exposure over the Archives of Sexual Behavior, 43(6), 1187–1197. https://doi.org/10.1007/s10508-013- life course and the severity of sexual offenses: Imitation and cathartic effects. Journal 0115-z. of Criminal Justice, 40(1), 21–30. Short, M. B., Black, L., Smith, A. H., Wetterneck, C. T., & Wells, D. E. (2012). A review of Marshall, E. A., Miller, H. A., & Bouffard, J. A. (2017). Crossing the threshold from porn internet pornography use research: Methodology and content from the past 10 years. use to porn problem: Frequency and modality of porn use as predictors of sexually Cyberpsychology, Behavior and Social Networking, 15(1), 13–23. https://doi.org/10. coercive behaviors. Journal of Interpersonal Violence. Advanced online publication 1089/cyber.2010.0477. https://doi-org.ezproxy.shsu.edu/10.1177/0886260517743549. Short, M. B., Kasper, T. E., & Wetterneck, C. T. (2015). The relationship between re- Marshall, E. A., Miller, H. A., & Bouffard, J. A. (2018). Bridging the theoretical gap: Using ligiosity and internet pornography use. Journal of Religion and Health, 54(2), sexual script theory to explain the relationship between pornography use and sexual 571–583. https://doi.org/10.1007/s10943-014-9849-8. coercion. Journal of Interpersonal Violence. https://doi.org/10.1177/ Štulhofer, A., Buško, V., & Landripet, I. (2010). Pornography, sexual socialization, and 0886260518795170 Advanced online publication. satisfaction among young men. Archives of Sexual Behavior, 39(1), 168–178. https:// Mattebo, M., Tydén, T., Häggström-Nordin, E., Nilsson, K. W., & Larsson, M. (2016). doi.org/10.1007/s10508-008-9387-0. Pornography consumption among adolescent girls in Sweden. The European Journal of To, S., Iu Kan, S., & Ngai, S. S. (2015). Interaction effects between exposure to sexually Contraception & Reproductive Health Care, 21(4), 295–302. https://doi.org/10.1080/ explicit online materials and individual, family, and extrafamilial factors on Hong 13625187.2016.1186268. Kong high school students’ beliefs about gender role equality and body-centered Minarcik, J., Wetterneck, C. T., & Short, M. B. (2016). The effects of sexually explicit sexuality. Youth & Society, 47(6), 747–768. https://doi.org/10.1177/ material use on romantic relationship dynamics. Journal of Behavioral Addictions, 0044118X13490764. 5(4), 700–707. https://doi.org/10.1556/2006.5.2016.078. Tomaszewska, P., & Krahe, B. (2018). Predictors of sexual aggression victimization and Morrison, T. G., Morrison, M. A., & Bradley, B. A. (2007). Correlates of gay men’s self- perpetration among polish university students: A longitudinal study. Archives of reported exposure to pornography. International Journal of Sexual Health, 19(2), Sexual Behavior, 47(2), 493–505. https://doi.org/10.1007/10508-016-0823-2. 33–43. https://doi.org/10.1300/J514v19n02_03. Tomić, I., Burić, J., & Štulhofer, A. (2017). Associations between Croatian adolescents’ use Mossige, S., Ainsaar, M., & Svedin, C. (Eds.). (2007). The Baltic Sea regional study on of sexually explicit material and sexual behavior: Does parental monitoring play a (NOVA rapport 18/07)Oslo, Norway: Norwegian Social Research. role? Archives of Sexual Behavior. https://doi.org/10.1007/s10508-017-1097-z. Retrieved from: http://www.reassess.no/index.gan?id. Traeen, B., & Daneback, K. (2013). The use of pornography and sexual behavior among Muusses, L. D., Kerkhof, P., & Finkenauer, C. (2015). Internet pornography and re- men and women of differing . Sexologies: European Journal of lationship quality: A longitudinal study of within and between partner effects of Sexology and Sexual Health, 22(2), 41–48. adjustment, sexual satisfaction and sexually explicit internet material among newly- Traen, B., Hald, G. M., Noor, S. W., Iantaffi, A., Grey, J., & Rosser, B. R. S. (2014). The weds. Computers in Human Behavior, 45,77–84. https://doi.org/10.1016/j.chb.2014. relationship between use of sexually explicit media and sexual risk behavior in men 11.077. who have sex with men: Exploring the mediating effects of sexual self-esteem and Nelson, K. M., Pantalone, D. W., Gamarel, K. E., & Simoni, J. M. (2016). A new measure of condom use self-efficacy. International Journal of Sexual Health, 26(1), 13–24. https:// the perceived influence of sexually explicit online media on the sexual behaviors of doi.org/10.1080/19317611.2013.823900. men who have sex with men. Journal of Sex Research, 53(4/5), 588–600. https://doi. Træen, B., Noor, S.W., Hald, G.M., Rosser, B.R.S., Brady, S.S., Erickson, D., … Wilkerson, org/10.1080/00224499.2015.1066744. J. M. (2015). Examining the relationship between use of sexually explicit media and Patterson, R., & Price, J. (2012). Pornography, religion, and the happiness gap: Does sexual risk behavior in a sample of men who have sex with men in Norway. pornography impact the actively religious differently? Journal for the Scientific Study Scandinavian Journal of Psychology, 56(3), 290–296. doi:https://doi.org/10.1111/ of Religion, 51(1), 79–89. https://doi.org/10.1111/j.1468-5906.2011.01630.x. sjop.12203. Peter, J., & Valkenberg, P. M. (2009). Adolescents’ exposure to sexually explicit internal Tripodi, F., Eleuteri, S., Giuliani, M., Rossi, R., Livi, S., Petruccelli, I., ... Simonelli, C. material and notions of women as sex objects: Assessing causality and underlying (2015). Unusual online sexual interests in heterosexual Swedish and Italian uni- processes. Journal of Communication, 59(3), 407–433. versity students. Sexologies: European Journal of Sexology and Sexual Health, 24(4), Peter, J., & Valkenburg, P. M. (2010). Processes underlying the effects of adolescents’ use e84–e93. https://doi.org/10.1016/j.sexol.2015.03.003. of sexually explicit internet material: The role of perceived realism. Communication Tylka, T. L. (2015). No harm in looking, right? Men’s pornography consumption, body Research, 37(3), 375–399. https://doi.org/10.1177/0093650210362464. image, and well-being. Psychology of Men & Masculinity, 16(1), 97–107. https://doi. Peter, J., & Valkenburg, P. M. (2011). The use of sexually explicit internet material and its org/10.1037/a0035774. antecedents: A longitudinal comparison of adolescents and adults. Archives of Sexual Vandenbosch, L., & Oosten, J. M. F. (2018). Explaining the relationship between sexually Behavior, 40(5), 1015–1025. https://doi.org/10.1007/s10508-010-9644-x. explicit internet material and casual sex: A two-step mediation model. Archives of Peter, J., & Valkenburg, P. M. (2012). Do questions about watching internet pornography Sexual Behavior. https://doi.org/10.1007/s10508-017-1145-8. make people watch internet pornography? A comparison between adolescents and Wetterneck, C. T., Burgess, A. J., Short, M. B., Smith, A. H., & Cervantes, M. E. (2012). adults. International Journal of Public Opinion Research, 24(3), 400–410. https://doi. The role of sexual compulsivity, impulsivity, and experiential avoidance in internet org/10.1093/ijpor/edr052. pornography use. The Psychological Record, 62(1), 3–18. Rasmussen, E. E., Rhodes, N., Ortiz, R. R., & White, S. R. (2016). The relation between Wijaya Mulya, T., & Hald, G. M. (2014). Self-perceived effects of pornography con- norm accessibility, pornography use, and parental mediation among emerging adults. sumption in a sample of Indonesian university students. Media Psychology, 17(1), Media Psychology, 19(3), 431–454. https://doi.org/10.1080/15213269.2015. 78–101. https://doi.org/10.1080/15213269.2013.850038. 1054944. Williams, K. M., Cooper, B. S., Howell, T. M., Yuille, J. C., & Paulhus, D. L. (2009). Rasmussen, K., & Bierman, A. (2016). How does religious attendance shape trajectories of Inferring sexually deviant behavior from corresponding fantasies: The role of per- pornography use across adolescence? Journal of Adolescence, 49, 191–203. https:// sonality and pornography consumption. Criminal Justice and Behavior, 36(2), doi.org/https://doi.org/10.1016/j.adolescence.2016.03.017. 198–222. https://doi.org/10.1177/0093854808327277. Rechter, E., & Sverdlik, N. (2016). Adolescents’ and teachers’ outlook on leisure activities: Willoughby, B. J., & Busby, D. M. (2016). In the eye of the beholder: Exploring variations Personal values as a unifying framework. Personality and Individual Differences, 99, in the perceptions of pornography. Journal of Sex Research, 53(6), 678–688. https:// 358–367. https://doi.org/10.1016/j.paid.2016.04.095. doi.org/10.1080/00224499.2015.1013601. Regnerus, M., Gordon, D., & Joseph, P. (2016). Documenting pornography use in Willoughby, B. J., Young-Peterson, B., & Leonhardt, N. D. (2018). Exploring trajectories America: A comparative analysis of methodological approaches. The Journal of Sex of pornography use through adolescence and emerging adulthood. Journal of Sex Research, 53(7), 873–881. https://doi.org/10.1080/00224499.2015.1096886. Research, 55(3), 297–309. Romito, P., & Beltramini, L. (2011). Watching pornography: Gender differences, violence Wolak, J., Mitchell, K., & Finkelhor, D. (2007). Unwanted and wanted exposure to online and victimization. Violence Against Women, 17, 1313–1326. pornography in a national sample of youth internet users. Pediatrics, 119, 247-225. Romito, P., & Beltramini, L. (2015). Factors associated with exposure to violent or de- Wright, P. J., Bae, S., & Funk, M. (2013). United States women and pornography through grading pornography among high school students. The Journal of School Nursing, four decades: Exposure, attitudes, behaviors, individual differences. Archives of Sexual 31(4), 280–290. Behavior, 42(7), 1131–1144. https://doi.org/10.1007/s10508-013-0116-y. Sabina, C., Wolak, J., & Finkelhor, D. (2008). The nature and dynamics of internet por- Wright, P. J., Sun, C., & Steffen, N. (2018). Pornography consumption, perceptions por- nography exposure for youth. CyberPsychology and Behavior, 11, 691–693. nography as sexual information, and condom use. Journal of Sex & Marital Therapy. Seigfried, K. (2007). Self-reported online child pornography behavior: A psychological ana- https://doi.org/10.1080/0092623X.2018.1462278. lysis. Unpublished master’s thesisNew York: John Jay College of Criminal Justice. Wright, P. J., Sun, C., Steffen, N. J., & Tokunaga, R. S. (2015). Pornography, alcohol, and Seigfried-Spellar, K. C. (2014). Distinguishing the viewers, downloaders, and exchangers male sexual dominance. Communication Monographs, 82(2), 252–270. https://doi. of internet child pornography by individual differences: Preliminary findings. Digital org/10.1080/03637751.2014.981558. Investigation, 11(4), 252–260. https://doi.org/10.1016/j.diin.2014.07.003. Wright, P. J., & Tokunaga, R. S. (2016). Men’s objectifying media consumption,

178 E.A. Marshall and H.A. Miller Aggression and Violent Behavior 48 (2019) 169–179

objectification of women, and attitudes supportive of violence against women. of Research & Treatment (Sage), 26(6), 546–568. https://doi.org/10.1177/ Archives of Sexual Behavior, 45(4), 955–964. https://doi.org/10.1007/s10508-015- 1079063213503688. 0644-8. Ybarra, M. L., Mitchell, K. J., Hamburger, M., Diener-West, M., & Leaf, P. J. (2011). X- Wright, P. J., Tokunaga, R. S., & Kraus, A. (2016). A meta-analysis of pornography rated material and perpetration of sexually aggressive behavior among children and consumption and actual acts of sexual aggression in general population studies. adolescents: Is there a link? Aggressive Behavior, 37(1), 1–18. https://doi.org/10. Journal of Communication, 66(1), 183–205. https://doi.org/10.1111/jcom.12201. 1002/ab.20367. Wryobeck, J. M., & Wiederman, M. W. (1999). Sexual narcissism: Measurement and Zheng, L., Zhang, X., & Feng, Y. (2017). The new avenue of online sexual activity in correlates among college men. Journal of Sexual and Marital Therapy, 25, 321–331. China: The smartphone. Computers in Human Behavior, 67, 190–195. https://doi.org/ Wurtele, S. K., Simons, D. A., & Moreno, T. (2014). Sexual interest in children among an 10.1016/j.chb.2016.10.024. online sample of men and women: Prevalence and correlates. Sexual Abuse: A Journal

179