Cross-Cultural Research Collaborations: Information for Collaborators Chulalongkorn University, Bangkok (Thailand) https://osf.io/z3xy4/ With the aim of improving the generalisability of psychological research, we are seeking to try and remove (some of) the barriers that might typically inhibit or demotivate researchers from stepping outside of WEIRD (Western, Educated, Industrialized, Rich, Democratic) society samples and engaging in cross-cultural collaborations.
Teaching students in a non-WEIRD (Thai) society has been a revelation in appreciating how many theories describing ‘human behaviour’ require a cultural disclaimer and I think we need to do much more to understand the applicability of theories in different cultures. However, it’s often far from straightforward to conduct cross-cultural research because of the difficulty in accessing heterogeneous samples of participants, particularly in the absence of existing collaborations or institutional links, (this is perhaps especially true for postgraduates or ECRs).
In response, we would like to invite anyone who is interested in engaging in cross- cultural data collection to contact us and discuss collaboration - we would be particularly keen to hear from any researchers who have effects from data collected using participants in a WEIRD society and would like us to perform a direct replication of their experiment as an additional test of the effect.. Ideally, this could be as simple as you sharing the code for your experimental program and we’ll set it up and collect the data for you. At the outset, we have one highly conscientious and motivated research assistant who is available to work on this part time; hopefully more funding will be available for this depending on interest.
1. Background and Motivation The Faculty of Psychology at Chulalongkorn University has a rich history of cross- cultural collaborations with affiliations to institutions in the US, UK, Australia, and Japan. Consequently, we have a wide range of ongoing research collaborations between our staff and members of faculty at affiliated Universities; typically involving collaboration at each stage of the research cycle: e.g., theorising, planning, data collection, analysis, writing, editing, etc. These kinds of cross-cultural collaborations where there exists mutual responsibility at each stage have considerable merit and are useful for a wide range of reasons. However, we also believe there is scope for developing a different type of collaborative approach that instead seeks to directly address two of the primary issues facing psychology as a science: generalisability and reproducibility.
1.1. Issues of Generalisability “Growing attention has been drawn to the lack of diversity in psychological testing, in particular to the fact that the vast majority of psychological research has been conducted on populations that are unrepresentative of human culture more globally—those from WEIRD (Western, educated, industrial, rich, and democratic) backgrounds (Henrich et al., 2010; Legare and Harris, 2016; Nielsen and Haun, 2016). The dearth of systematic research outside of Western cultural contexts is a major impediment to theoretical progress in the psychological sciences (Legare and Nielsen, 2015 ; Rowley and Camacho, 2015). Where psychological researchers assume that data are not specific to the sample of participants under direct test (i.e., that findings are generalizable), lack of attention to cultural variation and its psychological consequences risks yielding incomplete, and potentially inaccurate, conclusions (e.g., Apicella and Barrett, 2016; Evans and Schamberg, 2009; Mani et al., 2013 ; Votruba-Drzal et al., 2016).” (Nielsen, Haun, Kärtner, & Legare, 2017, pp. 32):
The quote above from Nielsen and colleagues effectively expresses our own perspectives on the present issues of diversity and generalisability inherent in psychological research. However, despite acknowledgment of the issues of generalisability and the use of WEIRD participants, there has been a widespread reluctance for researchers to engage in cross-cultural research or test their theories in heterogeneous samples. This reluctance is perhaps best understood through limited opportunities and incentives.
1.1.1. Opportunities: A major barrier to cross-cultural research is that it is not straightforward to develop collaborations that facilitate access to culturally heterogeneous samples and developing such collaborations is likely to take considerable time and resources whereby relationships are forged over a long time period. Thus, for PhD students or labs that do not have a strong relationship with researchers in another culture, there is often not the time or opportunity to develop the necessary links that might facilitate cross-cultural research.
1.1.2. Incentives: Put simply, researchers have few incentives to collect data from a heterogeneous sample. Journals have generally expressed only a modest preference for publishing data that includes non-WEIRD participants. Understandably, when researchers have limited time and resources, testing their theories in another sample is quite a low priority.
1.2. Issues of Reproducibility Following the well-publicised issues surrounding reproducibility in psychology (Open Science Collaboration, 2015), there is a growing momentum towards the potential benefits of multi-site collaborative projects for improving statistical power and transparency in research (Munafò et al., 2017). For example, if a researcher is to have sufficient power to detect a small or moderate size effect (depending on the nature of the design), this might require a sample size that is too large to feasibly collect all the data from one lab, whereas it may be achieved instead by involving many labs in a distributed data collection process. Similarly, this ‘Team Science’ approach increases the level of transparency in the data collection process, as well as allowing for greater understanding of the effects of culture.
Through adopting a structured approach to facilitating collaborations, we can go some way to make these opportunities available for researchers, at least for those individuals who are sufficiently motivated. With respect to the issue of incentives, these are admittedly rather limited in the present climate. However, we strongly feel that we are reaching a stage where genuine shifts are being made towards “(a) encouraging publication of studies that feature non-WEIRD participants, (b) encouraging replication in a new population of a previously established finding, and (c) encouraging theoretically motivated cross-cultural comparisons” (Nielsen et al., 2017, pp. 35). Provided the opportunity is there, we think the incentives will only increase of the coming years.
2. Data Collection Collaboration 2.1. Collaboration Process 1. Read the OSF page to get the latest details of the project: https://osf.io/z3xy4/ 2. Contact us via email ([email protected]) and provide details of your research study you would like us to collaborate on; including details of: sample size, demographics of participants, and summary of methods. For full details of what to provide see the link here. 3. We consider feasibility of data collection based on the number of participants required, time frame, availability of research assistants, equipment, etc. a. If feasible, we accept and assign a lead researcher based on their expertise in research area. b. We reject if not feasible 4. You share the relevant experimental materials (e.g., code, questionnaires, etc.) and a fully detailed description of the study procedures and protocol. 5. We perform any relevant translations of questionnaires and study information. 6. If you do not have IRB approval that covers our data collection, we will submit the study details to our office of ethics review committee for research involving human research subjects. 7. We pilot test the experiment for qualitative feedback 8. We collect the data. 9. We share the data with you. 10. We will assist in writing the manuscript and offering relevant cultural perspectives.
Note. To facilitate transparency in the research process, all experimental details and anonymised data will be shared to a project page hosted on OSF.io. If you would prefer that these were kept private until after publication we can discuss this.
2.2. Who We Want to Collaborate With? We are keen to hear from any researchers who are interested in collaborating with us to collect data. At the outset, we are especially keen to hear from researchers who already have data (from at least a single study) showing an effect (or null effect) in a WEIRD society and would like to extend and replicate their study design in a sample of Thai participants. Whilst we are in the initial stages, we would also prefer to collaborate on research that is reasonably straightforward to implement. For example, we would particularly encourage researchers who have replicable experimental code that can be setup to run on our computers, or with researchers who are interested in questionnaire measures.
2.3. What Do We Get Out of The Collaboration? If you publish the research with the data we have collected, we would ask for the lead researcher based in the Faculty of Psychology at Chulalongkorn University to be included as a co-author on the paper. If you decide not to publish the data, we would like to have the option to publish any outcomes ourselves; the specifics of this can be discussed on a case by case basis.
3. Further Details and Questions.
3.1. Our Location and Thai Culture. We are ideally located in the heart of Bangkok, Thailand; with a large student population in the surrounding campus (~ 33,000) and near some of the busiest tourist areas of Bangkok. With respect to Thai culture, an informative comparison between Thai and Western cultures can be gained from referencing geert-hofstede.com.
Figure 1. Comparison of Thailand with United States and United Kingdom According to the Hofstede’s Six-Dimensional Model. Figure adopted from https://geert- hofstede.com/thailand.html
As an overview, Thailand has a strong collectivist culture where the official religion is Theravada Buddhism (93.6%). There is a strong sense of awareness of hierarchy, and individuals have a great respect for traditions.
In addition to these general cultural perspectives, there are also some additional considerations that may be relevant: - Thailand was in a three-month period of mourning since the passing of the King in October 2016. Almost all citizens wore black clothing during the mourning period, and government officials and office workers have generally been instructed to wear black or white clothing until October 2017. - Dark black and purple are considered fortunate because of their association with royalty. - Numbers 13 and 4 are often perceived as unlikely. - The number 5 is associated with laughter (five is pronounced ‘ha’)
3.2. Facilities Our facilities are generally rather modest but we are well-equipped for conducting computer-based experiments. I'm familiar coding with Matlab, PsychoPy (Python) and R for making any necessary edits to ensure experiments are setup correctly on our monitors.
Software:
• PsychoPy • Inquisit v.4 (4 Computers) • E-Prime
Lab Facilities
• Two medium sized lab rooms with one-way mirrors for observation. • Variety of lab rooms with multiple computers
Hardware
• Psychophysiology and Biofeedback Lab o Skin Conductance (GSR) o Heart Rate Variability (HRV) o Electromyography (EMG) o Temperature Sensor
3.3. Translating text from English to Thai Researchers in the Faculty of Psychology department our highly experienced in performing English-Thai translations and back translations according to establish procedures (Brislin, 1970).
3.4. Subject Pool Composition: Depending on the requirements of the study, we can test the following participants in descending order of difficulty of access1: Thai Psychology students at Chulalongkorn University
1 Note. We are also ideally located to access a thriving backpacker population and in the future, we have research projects that will involve this demographic. If you are interested in asking a research question as part of any upcoming projects, please get in touch. Bilingual (English-Thai) students studying Psychology at Chulalongkorn University Thai Non-psychology students at Chulalongkorn University Thai Adults (i.e. non-students) from Bangkok province
References Brislin, R. W. (1970). Back-Translation for Cross-Cultural Research. Journal of Cross- Cultural Psychology, 1(3), 185–216. http://doi.org/10.1177/135910457000100301 Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie, N., … Wagenmakers, E. (2017). A manifesto for reproducible science. Nature Human Behavior, 1(January), 1–9. http://doi.org/10.1038/s41562-016-0021 Nielsen, M., Haun, D., Kärtner, J., & Legare, C. H. (2017). The persistent sampling bias in developmental psychology: A call to action. Journal of Experimental Child Psychology, 162, 31–38. http://doi.org/10.1016/j.jecp.2017.04.017 Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716-aac4716. http://doi.org/10.1126/science.aac4716