Part II Critical Issues in the Implementation of OSBPs Chapter 5 Baseline Surveys, Impact Assessment, and Monitoring for OSBPs 5.1 Introduction: Process of Baseline Surveys and Periodic Monitoring Figure 5-1 presents the process of carrying out surveys, monitoring, and studies required for the planning and operation of OSBPs, with with cross-references to sections and subsections of this chapter. Figure 5-1: Process of Conducting Surveys, Monitoring, and Studies for the Planning and Operation of OSBPs Source: This Sourcebook Figure 5-2 presents the timeline and purpose of carrying out each survey or study. In the planning phase, baseline surveys should be implemented to collect data for traffic demand forecasting and economic analysis. These studies are essential to design OSBP facilities of an appropriate size and layout and to assess the economic viability 1 of OSBP projects before proceeding with their implementation. Without careful assessment at this stage, investments in 1 Although this chapter focuses on the economic benefits of OSBP projects, an OSBP may be established for non- economic reasons (e.g., the planned Unity Bridge OSBP between Mozambique and Tanzania). 5-1 OSBPs might result in little or no benefits at the border crossing. After completion and operationalization of an OSBP, it is recommended to conduct endline/impact assessment surveys for project evaluation. A comparison of endline data with baseline data will make it possible to determine the benefits from implementing the project. Presenting such evidence is important for accountability. Monitoring can be undertaken periodically or continuously to record performance indicators on the operation of the OSBP. This exercise provides feedback for improving operations to realize better performance. Figure 5-2: Surveys, Impact Assessment, and Monitoring for OSBP Projects Timeline Survey/Monitoring Analysis Purpose National / Regional / To identify a need for the OSBP from a Corridor Level Analysis broader perspective Planning Baseline Surveys Traffic Demand To design OSBP facilities of an Forecasting appropriate size and layout To appraise the economic viability of the Economic Analysis OSBP project Design & Construction Monitoring Indicator Assessment To obtain feedback for improvement Endline / Impact To measure impacts and to obtain Project Evaluation Assessment feedback for improvement Operation Source: This Sourcebook The following sections and subsections present the details of each survey and analysis method. 5.2 Pre-Design (Planning) Studies 5.2.1 Types of Studies/Surveys Pre-construction studies based on data analysis are essential not only to design OSBP facilities of an appropriate size and layout, but also to assess the economic viability of an OSBP project before proceeding with construction and operationalization. Without careful assessment at this stage, investment in the project might result in little benefit. In this regard, traffic demand forecasting and economic analysis are fundamental studies to be carried out in the pre- construction stage. Before conducting surveys, the objectives, scope, and type of data to be collected should be clearly specified. The appropriate survey method will depend on the objective, which can be classified by geographic scope: (i) national/regional level; (ii) corridor level; or (iii) border post level. 5-2 In most cases of OSBP project planning, the border post level survey will provide concrete and fundamental information, while broader-level surveys are also important for strategic planning prior to the appraisal of specific OSBP projects. The types of data to be collected will also be considered in this section by their respective scope. 5.2.2 Indicators2 (1) Overview Key performance indicators (KPIs) are a tool for structured monitoring of OSBP operations. Before proceeding with facility development or operations, the implementers of an OSBP should prepare a monitoring plan to determine which indicators should to be tracked and how. A fair number of indicators should be selected from the viewpoint of technical measurability, the cost of data collection, relevance to the project purpose, specificity, and the consistency of measurement. Major categories of KPIs include indicators of traffic, time, facilitation/ procedures, and administration, each of which is discussed in the following subsections. (2) Indicators of Traffic According to classical theory, the volume of trade is a function of the economic size of the trading partners and inversely related to the disutility of transport and other trade costs between them. Therefore, improvements in corridor performance should be reflected in trade volumes along that border, and trade and transport volumes at a border crossing can be regarded as a reference indicator. Traffic data can be obtained from clearance data, which is routinely recorded. Traffic of international trade, local movements, and passenger movements should be monitored. Depending on the focus of the monitoring, disaggregation of the data is possible and desirable by type of cargo (e.g., containerized goods, general cargo, liquid bulk, and dry bulk), by direction, and by country (tons, TEUs), as well as by mode of transport, if there are options. (3) Indicators of Time Indicators of time – one of the most important categories of indicators – show average, median, maximum, and minimum times at a border post. They can be aggregated or broken down between different procedural steps. It is simple to collect this data with a time measurement survey, but it requires a good sampling and measurement methodology. Typically, these surveys include measures of waiting times before reaching the border station and the time spent there after release. For example, a set of specific indicators of time might include the following: (i) the time of truck transport from arrival at the OSBP to departure from 3 the OSBP (minutes, by pre-declared or not pre-declared); (ii) extra time for inspections of cargo with risk (minutes); and (iii) the time required for border crossing, for passenger vehicles/buses/motorcycles/others (minutes). 2 This section draws on: (i) Organization for Security and Co-operation in Europe and United Nations Economic Commission for Europe, Handbook of Best Practices at Border Crossings – A Trade and Transport Facilitation Perspective, 2012, pp. 240–241; and (ii) Charles Kunaka and Robin Carruthers, Trade and Transport Corridor Management Toolkit, World Bank, 2014, pp. 126–133. 3 E.g., measurement may start at a point before 1,000 m from the entrance gate of OSBP and end at the point after 1,000 m from the exit gate of the OSBP in order to capture time spent outside of the OSBP. 5-3 Time indicators can include the reliability of transport, which is often more important to traders than the actual time. As much as possible, the measures of time should provide detail on the distribution around the mean. (4) Indicators of Facilitation and Procedures Indicators of facilitation and procedures monitor the status of execution of measures for trade and transport and streamlined procedures at a border post. These indicators may include the number of procedures or documents required and the inspection rate of high-risk cargo or passengers. If installed, ICT systems can record some indicators of this category in daily operations; otherwise, interviews or surveys on average time are required. (5) Indicators of Administration As a foundation to deliver services with good performance, the status of administration or management of an OSBP is also worth monitoring. Indicators of administration can include the number of border officials, revenue collected per border official, trade volume per border official, total administration cost/revenue collected, declarations per border official, the number of meetings of the joint working committee, and administrative and maintenance expenditures. Border agencies should keep at least some of this data. 5.2.3 National/Regional/Corridor Analysis (1) National/Regional Surveys At the national or regional level, the main objectives of surveys are to compare the trade competitiveness of logistics performance with that of other countries and to identify major constraints or opportunities for improvement from a broad perspective. This assessment provides the rationale for trade facilitation projects, including OSBPs. Information on the trade facilitation environment in a country or region can usually be obtained from published data sources including: (i) the UN Commodity Trade Statistics Database (Comtrade) database exports and imports by detailed commodity and partner country4; (ii) the Trade Analysis Information System (TRAINS) of the United Nations Conference on Trade and Development (UNCTAD) for data on imports, tariffs, para-tariffs, and nontariff measures at the national level5; (iii) the Logistics Performance Index (LPI) of the World Bank with various categories of sub-indices to measure the logistics performance of countries 6,7; and (iv) the Trade Facilitation Indicators (TFIs) of the Organization for Co-operation and Development, for data on the degree of implementing trade facilitation measures. Also, other international indices, 8 national statistics, and trade demand forecast data can be used for assessments at this level. 4 See http://comtrade.un.org/. 5 http://www.unctad.info/en/Trade-Analysis-Branch/Key-Areas/TRAINSWITS/. 6 http://lpi.worldbank.org/. 7 LPI is based on perceptions identified in interviews. 8 The Doing Business database (http://www.doingbusiness.org/)