Wallace, F., K. Williams, R. Towler, and K. McGauley. 2015. Innovative Camera Applications 105 for Electronic Monitoring. In: G.H. Kruse, H.C. An, J. DiCosimo, C.A. Eischens, G.S. Gislason, D.N. McBride, C.S. Rose, and C.E. Siddon (eds.), : Global Issues and Creative Solutions. Alaska Sea Grant, University of Alaska Fairbanks. http://doi. org/10.4027/fbgics.2015.06

© Alaska Sea Grant, University of Alaska Fairbanks

Innovative Camera Applications for Electronic Monitoring Farron Wallace, Kresimir Williams, and Rick Towler NOAA Fisheries, Alaska Center, Seattle, Washington, USA, [email protected]

Katy McGauley Alaska Groundfish Data Bank, Anchorage, Alaska, USA

Abstract Electronic monitoring has been shown to be an effective tool to meet a variety of fisheries monitoring objectives in compliance-based pro- grams. However, these systems have not been effective in delivering individual fish data similar to information collected by an observer. Development of new camera-based systems, methods, and tools is critical for collecting scientific data to inform management. A camera system being developed at the Alaska Fisheries Science Center greatly improves the functionality and addresses many of the limitations of electronic monitoring systems. This system provides the ability to automatically collect length measurements in addition to monitoring for compliance. System capacity to identify and automatically capture high quality (HD) stereo images of catch events, for efficient identification of fish to species or species group, is being actively developed. Because only images of individual catch events are stored and reviewed, post- processing and storage costs are reduced, facilitating data transfer and data management. Event-based image capture will also improve real time reporting, reducing lag times associated with current monitoring and post processing methods. Images will be linked to GPS information, allowing precise location of all species-specific catch. Spatial image tag- ging will enable mapping of high bycatch rate areas, potentially improv- ing future management strategies to lower bycatch. Lowering costs associated with the collection, transfer, storage, and analysis of event- based image data proposed here will allow greater coverage rates to a wider range of vessel types and sizes where it is impractical to place an observer. By leveraging the latest developments in computer vision, 106 Wallace et al.—Innovative Camera Applications cost-effective and timely extraction of scientific data from images will provide greater certainty for resource management and support sustain- able practices.

Introduction There has been a long history of exploring the use and efficacy of elec- tronic monitoring (EM) in Alaska fisheries. In 2002, the National Marine Fisheries Service (NMFS) began exploring the use of EM as a tool to ensure compliance with the use of seabird deterrence devices and as a management tool to identify seabirds caught on longline vessels (Ames et al. 2007). In 2004, the North Pacific Management Council (Council) assessed EM projects being used in fisheries (MRAG 2004), which was followed by an EM workshop with broad participation from various management agencies, the , the environmental community, and the industry developing monitoring technologies in Alaska fisheries (AFSC 2008). By 2010 ten EM projects were completed to assess the general efficacy of EM technology in Alaska commercial fisheries. Two of these projects were conducted onboard Pacific halibut longline vessels, including two studies conducted by the International Pacific Halibut Commission (IPHC) and NMFS to examine the accuracy of fishing effort and catch composition data collected by EM relative to the traditional at-sea observer method (Ames 2005, Ames et al. 2007). Building upon these studies, a third collaborative study was conducted by NMFS, IPHC, and Pacific States Marine Fisheries Commission to evaluate the potential of EM as an alternative tool to monitor bycatch on Pacific halibut longline vessels (Cahalan et al. 2010). Other studies evaluated the effectiveness of EM technology to monitor the discard of prohibited species catch (PSC) on a factory trawler (Haist 2008, McElderry et al. 2008a) and monitor and enumerate discard aboard rockfish catcher vessels in the Gulf of Alaska (McElderry et al. 2005, Bonney and McGauley 2008, McElderry et al. 2008b, Bonney et al. 2009). These studies have led to implementation of camera systems used to monitor bin sorting and catch weight determination using a flow scale in several Alaska fisheries to ensure that the observer sampling was unbiased. In recent years there has been a renewed interest in advancing EM in fisheries operating in the North Pacific following the restructuring of the North Pacific Groundfish Observer Program, leading to a strate- gic plan to integrate electronic reporting and EM into these fisheries (Loefflad et al. 2014). In 2013, the observer program began deploying observers randomly and into fishing sectors not previously observed, including many smaller vessels. Because accommodating an observer onboard small catcher vessels can be logistically difficult, the Council asked NMFS to develop an EM system for the small vessel (less than 57.5 Fisheries Bycatch: Global Issues and Creative Solutions 107 feet long) halibut and sablefish (IFQ) fisheries capable of estimating discard from this fleet as a cost-effective alterna- tive to an at-sea observer. Although EM has been shown to be an effective tool to satisfy a variety of management objectives, including catch monitoring in compliance-based programs such as that operating in British Columbia, Canada (Stanley et al. 2009, 2011), there are currently no EM programs implemented in US fisheries where data are used for fisheries man- agement purposes. These systems have not been proven effective for delivering detailed data sets comparable to information collected by an observer necessary for estimating discard. There are many challenges in extracting data for catch estimation from video streams including (1) ability of EM to identify length composition, species composition, and catch weight; (2) potentially large time delay between data collection and use; and (3) ability to collect high quality data continuously in a harsh environment. Although EM is perceived by many as a cost-effec- tive alternative to at-sea observer programs, cost is highly influenced by management objective, coverage extent, complexity of the image systems, and costs associated with retrieving, storing, and reviewing the video data. To date, there has been little innovation in EM systems. Advanced redesign of EM tools, coupled with application development to automate post processing, is critical for collecting scientific data to inform management and to reduce costs. New camera-based EM systems described here greatly improve the functionality and address many of the issues previously identified with EM systems.

Materials and methods Two new camera-based EM systems have been developed at the Alaska Fisheries Science Center that have the potential to lower costs, improve functionality, and improve data quality. Each system is designed spe- cifically to meet one of two key sampling needs in fisheries where all catch are brought onboard and in fisheries where catch is typically discarded at the rail. As is common with many EM camera-based units, these systems can integrate data from a suite of sensors, including GPS, hydraulic pressure, and drum rotation monitors to determine set and haul positions and collect effort data. Image collection can be started when sensor information indicates that haul back is occurring; other- wise cameras remain in standby mode. The systems developed by AFSC use machine vision GigE interface cameras controlled by a central pc computer. Machine vision cameras are built for industrial inspection, capable of continuous operation over a range of temperatures and mechanical stresses such as high vibration and impacts similar to that found in a operation. They are commonly used in 108 Wallace et al.—Innovative Camera Applications many industrial applications where product images are acquired and then analyzed to determine if the product meets specifications. The first system, a self-contained camera “chute” system, was designed to sample catch that are brought onboard. The unit consists of a single machine vision camera (Allied Vision GX1920), 4 high-intensity LED strobes (Smart Vision ODS75), and a processing computer (Intel i-7 dual core model ADLQM67PC from ADL Embedded Solutions USA; Fig. 1). As fish are placed into the chute they trigger the rapid collection of several images that are processed, and only the image containing a full view of the fish is retained. Using high power strobes ensures high quality imagery is maintained regardless of environmental conditions. A blue-colored polyvinyl chloride (PVC) background on the floor of the chute enables efficient automated processing of catch images for length in real time. Image processing leverages the color contrast between the fish and the background to efficiently extract the fish component of the image and estimate its length (Fig. 2). The fixed distance from target to camera and fixed focal length of the lens allow for simple estimates of length by scaling the image space dimensions (e.g., object length in pixels) to real world dimensions, accounting for both image distortion due to the lens elements and parallax due to angle of view. Accurate length measurement from images requires a custom algorithm to esti- mate the linear distance from the fish snout to the center of the caudal fin, equivalent to a fork length measurement taken by observers. This is accomplished by first normalizing the fish position in the image to align the primary fish axis horizontally, and then estimating a centered linear distance along the midline of the fish body. A pilot project was conducted to evaluate the chute approach for determining accurate fish lengths and deployment logistics aboard fishing vessels. Work was conducted aboard the F/VConstellation (F.J. O’Hara Corporation) in July 2014 in the Bering Sea, with individual Pacific halibut Hippoglossus( stenolepis) measured using a measuring board and then passed though the fish chute two times to compare performance for light coloration of the blind side and dark coloration of the eyed side of the flatfish body; 185 fish were measured. The second imaging system consists of a stereo-camera unit to monitor catch as it is being hauled in at the rail for fixed-gear hook- and-line vessels. Two cameras will be placed into a single housing to collect simultaneous paired images controlled by a central process- ing unit similar to the one described in the chute camera unit (Fig. 3). Stereo-camera methods can provide for highly precise measurements of length (Ruff et al. 1995, Harvey et al. 2002, Williams et al. 2010) where the targets vary in range and position from the camera. Length estima- tion requires that the cameras are calibrated, which is accomplished using the camera calibration toolbox, a freely available software built Fisheries Bycatch: Global Issues and Creative Solutions 109

Figure 1. Housing with (a) central processing computer and power supply (b) fish chute with camera, (c) camera and housing and (d) strobe light. 110 Wallace et al.—Innovative Camera Applications

Figure 2. Original image (top), processed image used for segmentation (center) and estimated length of fish (cm, bottom). Fisheries Bycatch: Global Issues and Creative Solutions 111

Figure 3. Illustration of stereo camera setup and position to capture images of fish during hauling. 112 Wallace et al.—Innovative Camera Applications in Matlab computing language (Bouguet 2008). Calibration and length measurement is described by Williams et al. 2010.

Results Pilot study data demonstrates that the chute camera system is capable of highly accurate length estimates. Of the 185 fish measured during the study, two exceeded the length of the chute and were not measured. Fig. 4 shows paired measurement comparisons between 183 Pacific halibut measured using a length board and automatically generated image- based length estimates. Mean difference between methods for individual fish was 0.07 cm, with a standard deviation of 0.85 cm. Seventy-nine percent of fish measured from images fell within 1 cm of the physical measurements. Error rates did not appear to be length-dependent. The system performed equally well regardless of which side of the halibut was presented (mean difference for individual fish = 0.13 cm).

Discussion Recent technological advances in high resolution digital cameras, camera interfaces such as USB 3 and GigE and access to freely avail- able software tools have now made it possible for fishery researchers to develop EM systems and image processing applications to automate data collection from images. Although these developments have greatly improved camera performance and increased the data collection capac- ity of image-based sampling it is clear from past studies that collection of consistent high quality digital images of catch will be necessary to support automated image processing and data collection from images for use in accurate bycatch estimation. Image quality for any EM system is largely dependent on a vessel operator’s ability to maintain the EM system to keep camera lenses clear of water drops or debris. Image quality is also affected by variable lighting conditions where glare from the lights or the sun while hauling fish obscures the images. Poor image quality not only makes it difficult to consistently identify catch it also makes it very challenging if not impossible to automate fish measurement and species identification. Because EM system performance and data quality of EM remains largely dependent upon vessel operators, the data stream cannot be regarded as independent, leading to perception problems that may hinder full- scale implementation of these systems. Improving EM system reliability and an independent flow of data will greatly add to the acceptance of technology to achieve responsible conservation and management of fishery resources. Self-contained camera systems for commercial and research fishing vessels have been shown to maintain high image quality of catch, and Fisheries Bycatch: Global Issues and Creative Solutions 113

Figure 4. Comparison of image-based length estimates of Pacific halibut (Hippoglossus stenolepis) using the chute system and physical length measurements. Both methods resulted in comparable length frequency distributions (a). Differences in length estimation methods did not appear to be length-dependent (b). Image- based measurements compared well in terms of accuracy (mean difference = 0.07 cm) and precision (SD difference = 0.85) (c) 114 Wallace et al.—Innovative Camera Applications support automation of length measurement and species identification (White et al. 2006). The self-contained camera chute system developed at the Alaska Fishery Science Center builds on this idea to support automatic capture of high quality images of individual fish in a man- ner compatible with automated image analysis and length estimation. Reduction in the volume of data resulting from event-based image capture systems will facilitate data storage on the vessel, improve data transfer rates, and reduce the burden of data storage and management both on the vessel and on land-based servers. Time stamped images linked to GPS information will allow precise location of species-specific catch. This will allow mapping of high bycatch rate areas, improving future management strategies to lower bycatch. Lower overall costs could extend coverage rates to a wider range of vessels types and sizes where it is impractical to place an observer. Preliminary results are encouraging in terms of fish length estima- tion, although further work will be needed for higher target densities and fish that may be bent or curved in the image data. Currently in development, the species classification software will also further extend the capabilities of the chute camera system. Although image quality from the stereo rail camera is vulnerable to the same issues as traditional EM camera systems, the use of stereo- camera technology offers a significant advantage over conventional single camera systems. A stereo pair of images can be used to accurately measure a fish regardless of fish orientation or distance from camera. Application of stereo camera methods to underwater fish measurement resulted in errors of less than 5% (Williams et al. 2010). A high degree of accuracy will be necessary since length can be used to infer species weight, a requirement for bycatch estimation. Obtaining fish length at the rail will also improve our ability to collect length from species not usually landed and therefore not sampled by observers due to size (sleeper and large skates), or for species that easily drop off at the rail such as giant grenadier. Cooperative research between the National Marine Fisheries Service, Pacific States Marine Fisheries Commission, and the fishing industry is currently under way. This research will compare several EM systems, define performance standards, advance application development, and determine feasibility for applying the technology to address specified needs for management. Our eventual goal is to process images in real time on the vessel, allowing for automated download of the data once a vessel returns to port. A new generation of self-contained EM camera systems capable of collecting scientific data on fishery impacts will provide greater certainty for resource management and support sus- tainable fishing practices. Fisheries Bycatch: Global Issues and Creative Solutions 115

Acknowledgments Camera system design was greatly enhanced by contributions from Craig Rose, Alaska Fisheries Science Center. The pilot study field test- ing was made possible by efforts of John Gauvin, of Alaska Cooperative, and the captain and crew of the F/V Constellation (F.J. O’Hara Corp.) in fabrication of the chute camera prototype and provid- ing vessel time and crew for the project.

References AFSC. 2008. Electronic Fisheries Monitoring Workshop Proceedings, July 29-30, 2008, Seattle, WA. Alaska Fisheries Science Center. 55 pp. http:// alaskafisheries.noaa.gov/npfmc/PDFdocuments/conservation_issues/ EMproceedings.pdf Ames, R.T. 2005. The efficacy of electronic monitoring systems: A case study on the applicability of video technology for longline fisheries manage- ment. International Pacific Halibut Commission Scientific Report 80. http://www.iphc.int/publications/scirep/SciReport0080.pdf Ames, R.T., B.M. Leaman, and K.L. Ames. 2007. Evaluation of video technology for monitoring of multispecies longline catches. N. Am. J. Fish. Manage. 27:955-964. Ames, R.T., G.H.Williams, and S.M. Fitzgerald. 2005. Using digital video moni- toring systems in fisheries: Application for monitoring compliance of seabird avoidance devices and seabird mortality in Pacific halibut long- line fisheries. NOAA Tech Memo NMFS-AFSC-152. www.afsc.noaa.gov/ Publications/AFSC-TM/NOAA-TM-AFSC-152.pdf Bonney, J., and K. McGauley. 2008. Testing the use of electronic monitoring to quantify at-sea halibut in the central Gulf of Alaska rock- fish fishery. Alaska Groundfish Databank. EFP 07-02 Final Report. http:// www.npfmc.org/wp-content/PDFdocuments/conservation_issues/EM_ Phase1508.pdf Bonney, J., A. Kinsolving, and K. McGauley. 2009. Continued assessment of an electronic monitoring system for quantifying at-sea discards in the central Gulf of Alaska rockfish fishery. Alaska Groundfish Databank. EFP 08-01 Final Report. http://alaskafisheries.noaa.gov/sustainablefisheries/ efp/efp08-01_halibutdiscards_rpt.pdf Bouguet, J.Y. 2008. Camera calibration toolbox for Matlab. http://vision. caltech.edu/bouguetj/calib_doc/index.html Buckley, K. 2005. Video monitoring aboard Bering Sea factory trawlers: A pilot study. Report prepared for North Pacific Research Board. 48 pp. http:// www.nprb.org/ Cahalan, J.A., B.M. Leaman, G.H. Williams, B.H. Mason, and W.A. Karp. 2010. Bycatch characterization in the Pacific halibut fishery: A field test of elec- tronic monitoring technology. NOAA Tech Memo NMFS-AFSC-213. 66 pp. 116 Wallace et al.—Innovative Camera Applications

Harvey, E., D. Fletcher, and M. Shortis. 2002. Estimation of reef fish length by divers and by stereo-video: A first comparison of the accuracy and preci- sion in the field on living fish under operational conditions. Fish. Res. 57:255-265. http://dx.doi.org/10.1016/S0165-7836(01)00356-3 Haist, V. 2008. Alaska Groundfish Data Bank study to evaluate an electronic monitoring program for estimating halibut discards: Statistical analy- sis of study data. Report prepared for the Alliance Foundation with funds from NOAA. http://www.marineconservation- alliance.org/wp-content/uploads/2013/03/FINAL_REPORT_EM_Analysis_ Halibut_Discards_March_2008_HAIST.pdf Loefflad, M.R., F.R. Wallace, J. Mondragon, J. Watson, and G.A. Harrington. 2014. Strategic plan for electronic monitoring and electronic reporting the North Pacific. NOAA Tech Memo NMFS-AFSC-276. 52 pp. http://www.afsc. noaa.gov/Publications/AFSC-TM/NOAA-TM-AFSC-276.pdf McElderry, H., R.D. Reidy, and D.F. Pahti. 2008a. A pilot study to evaluate the use of electronic monitoring on a Bering Sea groundfish factory trawler. International Pacific Halibut Commission Technical Report No. 51. http:// www.iphc.int/publications/techrep/tech0051.pdf McElderry, H., J. Schrader, and J. Illingworth. 2003. The efficacy of video-based monitoring for the halibut fishery. CSAS Research Document 2003/042. http://www.westcoast.fisheries.noaa.gov/publications/nepa/groundfish/ bycatch_eis/bycatch-eis-appd.pdf McElderry, H., R.D. Reidy, J. Illingworth, and M. Buckley. 2005. Electronic monitoring of the Kodiak, Alaska rockfish fishery, a pilot study. Report by Archipelago Marine Research Ltd., Victoria, BC and Digital Observer, Inc., Kodiak, AK, for NOAA. 43 pp. http://www.afsc.noaa.gov/FMA/PDF_ DOCS/2005kodiakemreport.pdf McElderry, H., J. Schrader, T. Wallin, and S. Oh. 2008b. Trials on F/V Sea Mac to evaluate the use of electronic monitoring for the Kodiak, AK Rockfish Pilot Program. Report by Archipelago Marine Research Ltd with Marine Conservation Alliance Foundation for NOAA. http://www.alaskamsf. org/wp-content/uploads/2013/02/FINAL_REPORT_Evaluate_use_of_ Electronic_Monitoring_for_rockfish_Pilot_Jan2008_Archipelago.pdf MRAG. 2004. Appendix I to the EA/RIR/IRFA for BSAI Amendment 86/GOA Amendment 76: Extension or Modification of the Program for Observer Procurement and Deployment in the North Pacific, public review draft, May 12, 2006. Appendix I: Fisheries Monitoring Technologies of a report prepared for the NPFMC by MRAG Americas, Inc., April 2004. Pria, M.J., H. McElderry, and A. Fedoruk. 2012. Estimating weight and identify- ing species through electronic monitoring (EM): A preliminary compari- son of electronic and observer based reporting. Interim technical review option period 1. Prepared for NOAA NEFSC. Pria, M.J., H. McElderry, S. Oh, A. Siddall, and R. Wehrell. 2008. Use of a video electronic monitoring system to estimate catch on groundfish fixed gear vessels in California: A pilot study. Report prepared for NMFS by Archipelago Marine Research Ltd., Victoria, BC, Canada. 46 pp. Fisheries Bycatch: Global Issues and Creative Solutions 117

Ruff, B.P., J.A. Marchant, and A.R. Frost. 1995. Fish sizing and monitoring using stereo image analysis system applied to . Aquat. Engineer. 14:155-173. http://dx.doi.org/10.1016/0144-8609(94)P4433-C Stanley, R.D., N. Olsen, and A. Fedoruk. 2009. The accuracy of yelloweye rock- fish catch estimates from the British Columbia groundfish integration project. Mar. Coast. Fish. 1:354-362. http://dx.doi.org/10.1577/C09-005.1 Stanley, R.D., H. McElderry, T. Mawani, and J. Koolman. 2011. The advan- tages of an audit over a census approach to the review of video imag- ery in fishery monitoring. ICES J. Mar. 68(8):1621-1627. Sci. http://dx.doi. org/10.1093/icesjms/fsr058 White, D.J., C. Svellingen, and N.J.C. Strachen. 2006. Automated measurement of species and length of fish by computer vision. Fish. Res. 80:203-210. http://dx.doi.org/10.1016/j.fishres.2006.04.009 Williams, K., C.N. Rooper, and R.H. Towler. 2010. Use of stereo camera systems for assessment of rockfish abundance in untrawlable areas and for record- ing pollock behavior during midwater trawls. Fish. Bull. 108:352-362.