sustainability Article An Item Response Theory to Analyze the Psychological Impacts of Rail-Transport Delay Mahdi Rezapour 1,*, Kelly Cuccolo 2, Christopher Veenstra 2 and F. Richard Ferraro 2 1 Wyoming Technology Transfer Center, University of Wyoming, Laramie, WY 82071, USA 2 Department of Psychology, University of North Dakota, Grand Forks, ND 58201, USA; [email protected] (K.C.); [email protected] (C.V.); [email protected] (F.R.F.) * Correspondence: [email protected] Abstract: Questionnaire instruments have been used extensively by researchers in the literature review for evaluation of various aspects of public transportation. Important implications have been derived from those instruments to improve various aspects of the transport. However, it is important that instruments, which are designed to measure various stimuli, meet criteria of reliability to reflect a real impact of the stressors. Particularly, given the diverse range of commuter characteristics considered in this study, it is necessary to ensure that instruments are reliable and accurate. This can be achieved by finding the relationship between the item’s properties and the underlying unobserved trait, being measured. The item response theory (IRT) refers to measurement of an instrument’s reliability by examining the relationship between the unobserved trait and various observed items. In this study, to determine if our instrument suffers from any potentially associated problems, the IRT analysis was conducted. The analysis was employed based on the graded response model (GRM) due to the ordinal nature of the data. Various aspects of the instruments, such as discriminability and informativity of the items were tested. For instance, it was found while the classical test theory Citation: Rezapour, M.; Cuccolo, K.; (CTT) confirm the reliability of the instrument, IRT highlight some concerns regarding the instrument. Veenstra, C.; Ferraro, F.R. An Item Also, the person fit assessment measure, for instance, highlights some concern regarding respondents Response Theory to Analyze the answering some of the questions due to lack of interest, choosing answers randomly. Not many Psychological Impacts of Rail- studies have examined instruments’ reliability in determining the psychological impacts of public Transport Delay. Sustainability 2021, transportation on commuters in the way that was performed here. 13, 6935. https://doi.org/10.3390/ su13126935 Keywords: item response theory; transport psychology; item information curve; the test information; residuals; transport delay Academic Editor: Tomio Miwa Received: 10 May 2021 Accepted: 17 June 2021 1. Introduction Published: 20 June 2021 Public transportation systems are a crucial part of society. Their benefits can be Publisher’s Note: MDPI stays neutral summarized as reduction in traffic congestion, carbon emission, and pollution-related with regard to jurisdictional claims in health concerns. However, public transport is not without shortcomings. Extensive efforts published maps and institutional affil- have been made by public transport engineers and planners to enhance public satisfaction. iations. This has been achieved, for instance, by improvements in specific factors such as the reliability of public transport (e.g., [1]), which could lead to greater use of public transport. A big concern that still has not received much attention is the negative aspects of public transport that could impact the well-being of commuters. However, despite many efforts to improve public transport, the services in many Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. areas are still not without shortcomings. One aspect of public transport that has received This article is an open access article substantial attention is its delay. Delay can be defined as part of waiting time, the difference distributed under the terms and between a service’s expected arrival time and its actual arrival time. Expectations within conditions of the Creative Commons commuters might be created by schedules that are presented by policy makers or by real- Attribution (CC BY) license (https:// time information displays. However, if these are not created, or no precise information is creativecommons.org/licenses/by/ provided, expectations might result from the past experiences of a commuter regarding the 4.0/). typical arrival and departure of a public transit service. Sustainability 2021, 13, 6935. https://doi.org/10.3390/su13126935 https://www.mdpi.com/journal/sustainability Sustainability 2021, 13, 6935 2 of 14 Despite the importance of delay on commuters’ satisfaction about public transport [2], sometimes delay is inevitable, so it is important to determine how delay is transferred to commuters in terms of various psychophysical behaviors and how those feelings impact the perceived quality of the system. To achieve that, researchers have designed instruments to measure commuters’ feel- ings and perceptions. However, those instruments are not without shortcomings. Item response theory (IRT) would help to validate these scales by highlighting many of the instrument challenges which need to be resolved. Despite the importance of validating instruments before their application, relatively little study has been done on the topic of transport psychology. 1.1. Study Motivation This study was conducted to evaluate and validate the instrument being used to measure the psychological impacts of delay on commuters. In this study, after using various IRT techniques of evaluating the reliability of the instrument, more measures were conducted to highlight the underlying causes of the lack of fit. For instance, person fit assessment was employed to flag those response observations that were not in line with all other observations. Also, a large portion of this manuscript is spent on mathematical formulation of the methodology. Despite the extensive efforts in studying and enhancing various aspects of public transport, not much study has been conducted evaluating the reliability of the instruments in that field. Thus, this study highlights several studies that were performed in other fields, implemented the IRT method. The next paragraph outlines a few studies conducted with the help of this technique. IRT was used for analyzing the parenting stress index for parents having children with autism spectrum disorder [3]. The results suggested that changes in distress severity were often reflected in an associated change in item score. However, other items functioned poorly for discriminating between criteria. In another study, IRT was used to examine cross- cultural comparability of standard scales of the occupational scales [4]. The IRT likelihood ratio model was used for differential item functioning (DIF), and differential test function- ing (DTF) analyses. The Rasch method was also used in different fields [5]. Depressive and anxiety symptoms in refugees were evaluated with the help of IRT [6]. The participants completed a patient health questionnaire related to depressive and anxiety symptoms. The results highlighted that interrelations of depressive and anxiety symptoms differed across residents and refugees. IRT was also implemented on the multidimensional assessment of parenting scale (MAPS); it confirmed that the positive and negative dimensions, and that the best-fitting model include six nested dimensions from the original model. IRT was used to measure the suitability of the author recognition test (ART) for native and nonnative English speakers [7]. The results showed an expected gradient. The autism- relevant quality of life in autistic adults was psychometrically evaluated [8]. IRT was also recently used for psychometric properties of Addenbrooke’s cognitive examination (ACE-III) [9], for self-reflection and insight scale [10], examination of the original and short-term difficulties in emotion recognition scales [11], and development and validation of the cancer knowledge scale for general population [12]. Despite the efforts on identifying translated impacts of delay on commuters, the question of whether the measurements possess the requisite psychometric properties remains to be answered. Sustainability 2021, 13, 6935 3 of 14 1.2. Problem Statement It is hypothesized that delay is not only a matter of reaching a destination, or the monetary value of expended time, but the psychological well-being of commuters. Even monetary valuation of time can be translated into psychophysical behaviors of individuals. Thus, the impacts of delay on commuters can be looked at from the perspective of its psychological implications for outcomes, such as stress or anxiety. For instance, stress could lead to serious illnesses including cardiovascular and suppressed immune functioning [13]. Various feelings can also result from delay occurrence alongside stress. These include feelings of anxiety, fear, or even anger. For instance, delay was highlighted as a primary factor of the stressfulness of traveling by car, especially during high traffic volume [14]. Crowding, delay, and accessibility to a railway station have also been shown to be sources of commuters’ anxiety [15]. A first step in addressing the huge cost of this stress is to understand the feelings that the respondent experiences regarding various associated stimuli. However, before conduct- ing any analysis to identify the factors, it is important for a developed instrument to be validated to make sure it is an accurate representation of commuters’ feelings. As such, this study was conducted to