Cross-Cultural Research Collaborations: Information For
Total Page:16
File Type:pdf, Size:1020Kb
Cross-Cultural Research Collaborations: Information for Collaborators Chulalongkorn University, Bangkok (Thailand) https://osf.io/z3xy4/ With the aim of improving the generalisability of psychological research, we are seeking to try and remove (some of) the barriers that might typically inhibit or demotivate researchers from stepping outside of WEIRD (Western, Educated, Industrialized, Rich, Democratic) society samples and engaging in cross-cultural collaborations. Teaching students in a non-WEIRD (Thai) society has been a revelation in appreciating how many theories describing ‘human behaviour’ require a cultural disclaimer and I think we need to do much more to understand the applicability of theories in different cultures. However, it’s often far from straightforward to conduct cross-cultural research because of the difficulty in accessing heterogeneous samples of participants, particularly in the absence of existing collaborations or institutional links, (this is perhaps especially true for postgraduates or ECRs). In response, we would like to invite anyone who is interested in engaging in cross- cultural data collection to contact us and discuss collaboration - we would be particularly keen to hear from any researchers who have effects from data collected using participants in a WEIRD society and would like us to perform a direct replication of their experiment as an additional test of the effect.. Ideally, this could be as simple as you sharing the code for your experimental program and we’ll set it up and collect the data for you. At the outset, we have one highly conscientious and motivated research assistant who is available to work on this part time; hopefully more funding will be available for this depending on interest. 1. Background and Motivation The Faculty of Psychology at Chulalongkorn University has a rich history of cross- cultural collaborations with affiliations to institutions in the US, UK, Australia, and Japan. Consequently, we have a wide range of ongoing research collaborations between our staff and members of faculty at affiliated Universities; typically involving collaboration at each stage of the research cycle: e.g., theorising, planning, data collection, analysis, writing, editing, etc. These kinds of cross-cultural collaborations where there exists mutual responsibility at each stage have considerable merit and are useful for a wide range of reasons. However, we also believe there is scope for developing a different type of collaborative approach that instead seeks to directly address two of the primary issues facing psychology as a science: generalisability and reproducibility. 1.1. Issues of Generalisability “Growing attention has been drawn to the lack of diversity in psychological testing, in particular to the fact that the vast majority of psychological research has been conducted on populations that are unrepresentative of human culture more globally—those from WEIRD (Western, educated, industrial, rich, and democratic) backgrounds (Henrich et al., 2010; Legare and Harris, 2016; Nielsen and Haun, 2016). The dearth of systematic research outside of Western cultural contexts is a major impediment to theoretical progress in the psychological sciences (Legare and Nielsen, 2015 ; Rowley and Camacho, 2015). Where psychological researchers assume that data are not specific to the sample of participants under direct test (i.e., that findings are generalizable), lack of attention to cultural variation and its psychological consequences risks yielding incomplete, and potentially inaccurate, conclusions (e.g., Apicella and Barrett, 2016; Evans and Schamberg, 2009; Mani et al., 2013 ; Votruba-Drzal et al., 2016).” (Nielsen, Haun, Kärtner, & Legare, 2017, pp. 32): The quote above from Nielsen and colleagues effectively expresses our own perspectives on the present issues of diversity and generalisability inherent in psychological research. However, despite acknowledgment of the issues of generalisability and the use of WEIRD participants, there has been a widespread reluctance for researchers to engage in cross-cultural research or test their theories in heterogeneous samples. This reluctance is perhaps best understood through limited opportunities and incentives. 1.1.1. Opportunities: A major barrier to cross-cultural research is that it is not straightforward to develop collaborations that facilitate access to culturally heterogeneous samples and developing such collaborations is likely to take considerable time and resources whereby relationships are forged over a long time period. Thus, for PhD students or labs that do not have a strong relationship with researchers in another culture, there is often not the time or opportunity to develop the necessary links that might facilitate cross-cultural research. 1.1.2. Incentives: Put simply, researchers have few incentives to collect data from a heterogeneous sample. Journals have generally expressed only a modest preference for publishing data that includes non-WEIRD participants. Understandably, when researchers have limited time and resources, testing their theories in another sample is quite a low priority. 1.2. Issues of Reproducibility Following the well-publicised issues surrounding reproducibility in psychology (Open Science Collaboration, 2015), there is a growing momentum towards the potential benefits of multi-site collaborative projects for improving statistical power and transparency in research (Munafò et al., 2017). For example, if a researcher is to have sufficient power to detect a small or moderate size effect (depending on the nature of the design), this might require a sample size that is too large to feasibly collect all the data from one lab, whereas it may be achieved instead by involving many labs in a distributed data collection process. Similarly, this ‘Team Science’ approach increases the level of transparency in the data collection process, as well as allowing for greater understanding of the effects of culture. Through adopting a structured approach to facilitating collaborations, we can go some way to make these opportunities available for researchers, at least for those individuals who are sufficiently motivated. With respect to the issue of incentives, these are admittedly rather limited in the present climate. However, we strongly feel that we are reaching a stage where genuine shifts are being made towards “(a) encouraging publication of studies that feature non-WEIRD participants, (b) encouraging replication in a new population of a previously established finding, and (c) encouraging theoretically motivated cross-cultural comparisons” (Nielsen et al., 2017, pp. 35). Provided the opportunity is there, we think the incentives will only increase of the coming years. 2. Data Collection Collaboration 2.1. Collaboration Process 1. Read the OSF page to get the latest details of the project: https://osf.io/z3xy4/ 2. Contact us via email ([email protected]) and provide details of your research study you would like us to collaborate on; including details of: sample size, demographics of participants, and summary of methods. For full details of what to provide see the link here. 3. We consider feasibility of data collection based on the number of participants required, time frame, availability of research assistants, equipment, etc. a. If feasible, we accept and assign a lead researcher based on their expertise in research area. b. We reject if not feasible 4. You share the relevant experimental materials (e.g., code, questionnaires, etc.) and a fully detailed description of the study procedures and protocol. 5. We perform any relevant translations of questionnaires and study information. 6. If you do not have IRB approval that covers our data collection, we will submit the study details to our office of ethics review committee for research involving human research subjects. 7. We pilot test the experiment for qualitative feedback 8. We collect the data. 9. We share the data with you. 10. We will assist in writing the manuscript and offering relevant cultural perspectives. Note. To facilitate transparency in the research process, all experimental details and anonymised data will be shared to a project page hosted on OSF.io. If you would prefer that these were kept private until after publication we can discuss this. 2.2. Who We Want to Collaborate With? We are keen to hear from any researchers who are interested in collaborating with us to collect data. At the outset, we are especially keen to hear from researchers who already have data (from at least a single study) showing an effect (or null effect) in a WEIRD society and would like to extend and replicate their study design in a sample of Thai participants. Whilst we are in the initial stages, we would also prefer to collaborate on research that is reasonably straightforward to implement. For example, we would particularly encourage researchers who have replicable experimental code that can be setup to run on our computers, or with researchers who are interested in questionnaire measures. 2.3. What Do We Get Out of The Collaboration? If you publish the research with the data we have collected, we would ask for the lead researcher based in the Faculty of Psychology at Chulalongkorn University to be included as a co-author on the paper. If you decide not to publish the data, we would like to have the option to publish any outcomes ourselves; the specifics of this can be discussed on a case by case basis. 3. Further Details and Questions. 3.1. Our Location and Thai