Technical Memoranda Reference Documents Classic Publications Contract/Grant Reports CMER Publications Series Information Links and Contacts Annual/Biennial Lists Web Manager Email Search Publications Publications Home Site Map

Introduction

Electronic Logbook Design

Data Analysis

Study Fleet Data Quality Review

Fleet Composition

Discussion and Conclusions

Acknowledgments

References Cited

NOAA Technical Memorandum NMFS NE 204

An Evaluation of the Northeast Region’s Study Fleet
Pilot Program and Electronic Logbook System:
Phases I and II

by Michael C. Palmer, Susan E. Wigley, John J. Hoey, and Joan E. Palmer

National Marine Fisheries Service, 166 Water Street, Woods Hole, MA 02543

Print publication date June 2007; web version posted January 23, 2008

Citation: Palmer MC, Wigley SE, Hoey JJ, Palmer, JE.  2007. An Evaluation of the Northeast Region’s Study Fleet pilot program and Electronic Logbook System: Phases I and II. NOAA Tech Memo NMFS NE 203; 79 p.

Download complete PDF/print version

ABSTRACT

A Study Fleet pilot program was initiated to: (1) assemble a “study fleet” of commercial New England groundfish vessels capable of providing high resolution (haul-by-haul) self-reported data on catch, effort and environmental conditions while conducting “normal” fishing operations; and (2) develop and implement an electronic data collection system.  An electronic logbook system (ELB) was developed and tested to collect, transfer and store data collected at sea by fishers.  Field testing and data collection was conducted on board a variety of groundfish vessels from November 2002 to August 2005 in a two-phased approach.  Approximately 1,100 trips were reported by 33 vessels using the ELB system during Phases I and II. Study Fleet vessels did not constitute a random sample and were not based on a statistical design. Instead, vessels were selected to test the feasibility of obtaining high quality self-reported catch data under realistic field conditions.

Study Fleet data were compared to existing fishery-dependent data collection programs used in the National Marine Fisheries Service Northeast Region to assess data quality and identify areas where improvements are needed.  Data were evaluated at four levels: trip, haul, species catch, and landings.  Overall, the Study Fleet data were similar to that collected by the Northeast Fisheries Observer Program (NEFOP) and offered increased accuracy and precision over the Fishing Vessel Trip Reports (FVTR) in terms of identifying the area of fishing and duration of effort.  Because of the electronic data collection and at-sea transmission of data, Study Fleet data are timelier than FVTR and NEFOP data.  Study Fleet data more accurately captured the statistical area fished than did the FVTR and were consistent with NEFOP data.   Species catch reporting was found to be generally consistent with other programs; however, the haul-by-haul nature of the Study Fleet pilot program resulted in increased reporting of discarded catch compared to FVTR data. Additional training is warranted for Study Fleet participants to reinforce the data reporting for each individual species and to improve hail weight estimations.  All vessel-based components of the technology used are reviewed with recommendations for continued use or modifications where appropriate.  The Study Fleet pilot program was successful in developing, testing, and deploying an ELB system among the New England groundfish fleet. Deployment of future Study Fleet data collection programs will depend upon the program objectives.

INTRODUCTION

Study fleets are a sample of vessels from a defined fleet that provide detailed, self-reported fisheries data for the purpose of addressing specific scientific needs.  Study fleets have been employed in the United States , including the New England region, and elsewhere around the world.  Recent study fleets have employed electronic data collection to improve the timeliness and precision of the data (Bucklin et al. 2001; Gallaway et al. 2003a; Gallaway et al. 2003b; Hendrickson et al. 2003).  In late 2000, workshops were conducted throughout New England involving the National Marine Fisheries Service (NMFS), industry representatives and nongovernment organizations (NGOs) to discuss industry interest in developing a New England groundfish Study Fleet.  The Study Fleet pilot program would have the dual objectives of: (1) assembling a “study fleet” of commercial New England groundfish vessels capable of providing high resolution (temporal and spatial) self-reported data on catch, effort and environmental conditions while conducting “normal” fishing operations; and (2) developing and implementing electronic reporting hardware and software for the collection, recording, and transferring of more accurate and timely fishery-based data (Gulf of Maine Aquarium 2001).

The program was intended to ultimately provide stock assessment scientists with more precise and accurate fishery-dependent data (e.g., more precise estimates of fishing effort, spatially explicit catch, and discard locations) and to improve the understanding of catch rates and species assemblages through examination of variables such as time of day, temperature, depth, tidal strength, and sediment type.  Additionally, the collaborative nature of the Study Fleet pilot program could create a channel through which stock assessment scientists and industry members could directly communicate and share information that would serve as the basis for future collaborative research projects (Murawski 2002).

In October 2002 Technology Planning and Management Corporation[1] was contracted to manage the Study Fleet pilot program.  In November 2002, Phase I commenced with a fleet size of approximately 15 paid participants; some participants operated multiple vessels. Phase I focused on developing the electronic logbook (ELB) software and testing supporting hardware.  During Phase I, there was typically an average of fewer than 10 vessels reporting a combined total of 20 trips per month (Figure 1).  Phase II, which began in September 2004, expanded the fleet size to 30 paid participants and continued testing and development of the electronic logbook technology, with particular emphasis on the area of satellite communications and refining the ELB.  By the end of Phase II in May 2005, two ELB systems had been developed and a Study Fleet of approximately 32 fishing vessels had been assembled (one vessel left the Program after Phase I).  Following the end of Phase II, the program entered a voluntary phase where participants were not compensated for data collection.  The voluntary phase ended August 31, 2005 with approximately 9 vessels reporting an additional 99 trips and providing additional information on software and hardware performance.

The idea of a groundfish study fleet arose from a need to improve the precision of data that scientists have typically extracted from mandatory Fishing Vessel Trip Reports (FVTRs).  Even so the data had to be collected in a manner that was more cost-effective than increasing the coverage of at-sea fisheries observers.  Current fishing vessel trip report (FVTR) regulations (50 CFR §648.7) require submission of paper logbooks on completion of a fishing trip.  The FVTR logbook captures two levels of reporting resolution: trip-level and subtrip-level[2].  Trip-level reporting captures information on vessel, operator, date/time of sail, crew size, the type of trip, date/time of landing, port of landing, and the dealer allocation of retained species.  Subtrip-level reporting captures the fishing gear used, gear configuration, number of hauls made, average duration of hauls, average location, statistical area in which fishing occurred, and average depth, as well as the resulting catch information (species, amount, and disposition).

FVTR regulations require that a separate logbook page be filled out for each statistical area fished and/or for each gear type/configuration (mesh size, etc.) fished. For each subtrip, the vessel is required to report the cumulative fishing effort.  For example, if a trawl vessel performs six hauls in a single statistical area with the same gear and mesh type, then the effort from the six hauls is averaged to determine an average haul location, duration, and depth, and the catch information from all six hauls is combined.

The subtrip data collection resolution is not sufficient to capture the fine scale catch and effort information needed for some analyses and stock assessments.  The collection of detailed information (e.g., gear configuration, duration, location, and timing) from individual units of fishing effort concurrent with catch attributes (e.g., species, amounts and disposition) can improve estimates of catch-per-unit effort (CPUE) and landings-per-unit-effort (LPUE).  For example, golden tilefish (Lopholatilus chamaeleonticeps) is assessed using a surplus production model which relies entirely on CPUE derived from FVTRs as an index of abundance.  The 2005 Stock Assessment Working Group (SARC 41) assumed that the length of a trip was directly related to effort because haul by haul information did not exist.  The days absent effort metric assumes that steam/search time are equal among trips.  Haul based effort and catch data are needed to improve the commercial CPUE index for this stock assessment (NEFSC 2005).

One of the goals of the Study Fleet pilot program was to have fishers record detailed data similar to that currently collected by the Northeast Fisheries Observer Program (NEFOP).  NEFOP trained sea-going observers are deployed aboard commercial vessels to collect detailed fisheries data.  For each observed haul, data are recorded on the gear characteristics (e.g., mesh size, mesh type, sweep length, mainline length, number of hooks, number of pots), fishing location (e.g., latitude and longitude, statistical area, fishing depth) and time (e.g., time of day, duration of effort) and the resulting catch by species and disposition (amount kept and/or discarded).

Unlike fisheries observers whose sole job is to record these data, the fishers must collect this information in addition to their normal fishing duties.  An ELB system can facilitate this potentially burdensome reporting process by automating much of the data entry.  Additionally, the logbook software can take advantage of satellite communication systems onboard the vessel, such as a vessel monitoring system (VMS) to transmit these data from sea because the fishers reported data have already been captured in electronic format within the logbook software.  Such technology could improve the timeliness of these data.  Currently, FVTRs must be submitted to the Northeast Regional Office (NERO) by the 15^th of the following month from which the trip landed (e.g. the reports of trips landing in October are due by November 15).  The delay between the end of the trip and receipt of the logbook by NMFS can be as great as 45 days.  When coupled with processing time and data entry into a relational database, it can be substantially longer before data are available to analysts.  On completion of an observed trip, data sheets are checked for accuracy by the observer and then submitted (typically within 48 hours) to the NEFOP for data editing and entry into a relational database. Data are not made available to scientists and managers until they have passed a rigorous auditing routine.

All vessel-based components of the technology that was developed and tested are reviewed with recommendations for continued use or modifications where appropriate.  Additionally, the reporting trends observed in Study Fleet data are compared to FVTR and NEFOP data collection programs.  A general data quality review of the collected data is presented with special attention to differences among the two ELB software programs developed.  Based on these analyses, recommendations are made to improve the Study Fleet data collection program.  Finally, because one of the original program objectives was to assemble a study fleet representative of the New England groundfish fleet, the composition of the assembled study fleet is compared to the both New England groundfish fleet and the entire New England fleet (all fisheries). The intent of this report is to summarize findings and offer recommendations for future deployments of a Study Fleet.  This report provides a comprehensive summary of Phases I and II of the Study Fleet pilot program and serves as the final Northeast Fisheries Science Center (NEFSC) review of this Pilot Program.

Electronic Logbook Design

This section provides an overview of the hardware and software tested and presents results and recommendations for future Study Fleet deployment.  The final ELB system deployed in Phase II included ELB software installed on a personal computer (PC), a global positioning system (GPS), a gear-mounted temperature probe, a VMS for at-sea data transmission, and a database system housed at the NERO (Figure 2).

PC Laptop

Six different PCs (both laptop and tablet) running either Microsoft® Windows® 2000 or XP were tested over the course of the program.  The computers ranged from inexpensive laptops (Dell^TM Inspiron 2600, Latitude D505, and Latitude C640) to more expensive ruggedized laptops (Walkabout^TM Hammerhead XRT) and ruggedized tablet computers (Brite® Computers Xplore iX104).  One of the more difficult problems with the use of computers in an open-wheelhouse environment on small vessels was sun glare off the computer screen.  Excessive glare made the screen difficult, and at times impossible, to view.  To counteract glare problems, a tablet PC with a screen specifically designed for use in bright environments (Motion Computing® M1300) was tested.  The Motion Computing® PC sufficiently addressed the glare problem; however, the manufacturer-supplied tablet stand lacked the stability to counteract engine vibration and sea conditions.  Use of the Brite® Computer was discontinued in Phase I because of excessive screen glare.  The one Walkabout^TM Hammerhead unit and one of the Motion Computing® computers suffered from processor failure during Phase II of the program.

 While the Dell^TM laptops are susceptible to glare problems, they were widely used in open-wheelhouse environments.  There were no reported failures of the Dell^TM laptops.  Some units experienced deployment periods of over two years in both open and dry wheelhouses.  There were at least two occasions when the Dell^TM laptops were resuscitated after exposure to water and ice.  Erratic computer behavior was reported on several of the units (both laptops and tablet computers).  It was later determined that most, if not all, of these issues were symptomatic of driver incompatibilities with the installed operating systems.  Based on the cost and long-term performance, inexpensive laptops offer the best PC solution for future deployments.  Minimum requirement specifications are outlined in Table 1.

Electronic Logbook Software

The centerpiece of the ELB system is the ELB software.  Three logbook software systems were developed and tested over the course of Phases I and II.  The first, the Thistle Box[3] was determined early in Phase I not to be a viable ELB system for the multispecies fishery fleets.  The Thistle system had originally been developed to record trap-based lobster catches, and fishers experienced difficulty attempting to enter numerous species records.  The other two PC-based logbook systems, the University of New Hampshire (UNH) logbook and P-Sea Windplot© (PSW) logbook, are both capable of capturing similar data elements and receiving input from temperature probes and transmitting data to the VMS unit. The UNH logbook was an extension of existing software designed as a prototype for an electronic vessel trip report (EVTR).  The PSW logbook utilized a popular navigation and plotting software package[4] used by many in the commercial fleet.  The UNH logbook was deployed on a greater proportion of the fleet (31 of 33 vessels) because it was further developed than the PSW logbook, having more user-friendly features, such as gear-specific species default lists and trip summary reports. Most importantly, initial testing indicated that the PSW logbook software was incapable of capturing fixed-gear fishing effort because of its inability to track simultaneous efforts.  While the UNH logbook can track multiple efforts simultaneously, it cannot disassociate fishing effort from fishing trips, which limits its use in some fixed gear fisheries where gear is set on one trip and hauled on a subsequent trip. During Phases I and II these fixed-gear fisheries (e.g., sink gillnet) were accommodated through manual entry of the set times and haul durations, a procedure contrary to the intent of an ELB which is to automate data entry to the extent possible.  Logbook version use over the duration of this project is detailed in Table 2.  Despite the functional shortcomings of either logbook, both ELBs captured all of the mandatory data elements required per FVTR regulations in addition to detailed information on fishing effort (gear type, characteristics, time, location, etc.), kept and discard species weights at a haul level, and water temperature (Table 3).

Global Position System (GPS) Unit

GPS units were connected via serial connections to the ELB laptop.  The ELB software used the GPS input to acquire accurate time and position information associated with the setting and hauling of fishing gear.  Several different GPS units were used in the project.  Whenever possible, the vessel’s existing GPS unit was used.  The GPS feed was split with a Y-splitter cable so the ELB could use the GPS feed without disrupting existing GPS needs (plotting software, etc.).  In instances where the vessel did not already have a GPS unit or did not wish for their existing unit to be used for the ELB system, a Garmin^TM GPS 36 TracPak unit was installed.  The only requirement of the GPS unit was that it was capable of receiving and transmitting the following National Marine Electronics Association (NMEA) 0183 American Standards Code for Information Interchange (ASCII) interface specifications) transmitted sentence(s): GPRMC, GPGGA, and GPVTR or GPRMC, GPVTG, GPGLL, and GPGSA.  During this project, no GPS-specific problems were encountered; all GPS units performed adequately.

Vessel Monitoring Systems (VMS) Unit

The UNH and PSW logbooks were capable of using the two VMS (Boatracs® and SkyMate®) approved for use in the NMFS Northeast Region during Phase I and II to transmit data files from sea.  The appropriate PC client software was installed on the Study Fleet computers:  Boatracs® Windows User Interface (WBUI) (V 5.1.5 and up) and SkyMate® Client Console (V 3.5 and up).

Both VMSs successfully transmitted ELB files from sea.  There were some apparent latency and nonreceipt issues associated with the SkyMate® systems, though it is unclear whether this was caused by SkyMate equipment installation problems or the satellite technology.  A land-based transmission test was performed for the two VMSs between May 16 and May 30, 2006. The exact same ELB data file was transmitted ten times at various times of the day during this period with each of the VMSs (total of 20 transmission attempted).  Boatracs® transmissions were quicker and more reliable.  SkyMate® experienced a 50% transmission success rate with an average receipt delay (time between sending and receipt of data file at the Regional Office) of 165.3 (± 133.1) minutes.  Comparatively, Boatracs® transmission success rate was 100% with an average receipt delay of 7.1 (± 2.4) minutes.  While similar statistics were not collected throughout Phase I and II, anecdotal evidence suggests that SkyMate® performance was better than that observed during the above test period, but not at the level observed using the Boatracs® VMS.

SkyMate® offers three different VMS payment plans to accommodate varying monthly character usage.  Boatracs® offers only a single payment plan (Table 4).  SkyMate® estimates that nonscallop VMS users will require 14,880 characters per month to meet VMS reporting requirements (information not available for Boatracs).  While SkyMate® may appear to be a less expensive option, data files transmitted through Boatracs® are on average > 50% smaller than the same file transmitted through SkyMate® (Figure 3).  To protect the integrity of the data collected by the Study Fleet pilot program, the ELB software converts the output data file (text file, .txt) to a password protected, compressed binary ZIP file (.zip).  Because SkyMate® requires transmitted data to be in ASCII format, the compressed file must then be converted to ASCII format with a UUencode program (.enc).  The ACSCII conversion increases the size of the original zip file.  Boatracs® can accommodate transmission of the ZIP file.

While transmission success may be improved by using a Boatracs® VMS, installation costs and vessel power requirements will affect the vessel operator’s choice of a VMS unit. The SkyMate® unit costs approximately $1599 plus installation and activation costs (http://www.skymate.com) compared to $3195 plus installation costs for the Boatracs® unit (Lauri Paul, Boatracs, 9155 Brown Deer Road, Suite 8, San Diego, CA 92121, March 27, 2006, pers. comm.). The Boatracs® Fisheries Mobile Communication Terminal/GPS (FMCT/G) draws approximately 5 amps when operating at 12 V[5]. Comparatively, the SkyMate® Communicator draws < 2.5 amps (less when not transmitting) when operating at 12 V (URL: http://www.skymate.com). Some of the smaller vessels operating in the Northeast Region are not equipped with batteries sufficient to meet the power demands of the Boatracs® unit when the vessel is not in operation.

Temperature Probe

A temperature probe was used in a limited capacity in Phase II.  Three vessels used the probes on a total of 219 hauls.  The ACR® Nautilus 85 temperature probe was the only probe tested.  Use of this probe in a marine environment proved problematic because the probe had to be opened to gain access to the data port.  Opening the probe exposed sensitive electronics to sea water making corrosion a significant problem on many of the probes deployed.  Some vessel captains concluded that O-ring failure was the ultimate cause of the corrosion.  While the temperature data collected by the probe appeared accurate, the recommendation for future probe selection is to use a temperature probe capable of data upload without having to be opened. There are many commercially available temperature/depth probes capable of optic data transfer. Use of these probes would solve the corrosion problem and expand environmental data collection to include the fishing depth of the gear.

Serial-to-USB Converter

GPS, VMS, and temperature probe inputs used in Phases I and II required a serial connection.  The majority of the laptops and tablets used in this project were limited to one serial port and two USB ports.  While only three communications (COM) ports were needed, at least one USB port is needed for the field technician’s flash drive to download data or install software updates.  Two different configurations were tested. The first employed three single serial-to-USB converters connected to a 4-port USB hub which is connected to one of the computer’s USB ports. Both Belkin® and Keyspan^TM serial-to-USB converters were used; however, the software driver for the Belkin® converter experienced incompatibility problems with Windows® XP. The second configuration used a Keyspan^TM 4-port serial-to-USB converter. This was a more direct setup involving only a single converter as opposed to the three converters and a USB hub required in the first configuration.  An additional benefit of this setup is the flexibility to add one more serial connection without an additional converter. No problems were encountered with the Keyspan^TM driver during Phases I and II.

Peripherals: Keyboard and Mouse

Many of the ELB hardware setups used external keyboards and mice/trackballs.  There was no standard configuration used for these peripherals; the setups varied by user preference and the peripheral inventory of the Study Fleet pilot program when vessels were being equipped with hardware.  Many of the smaller vessels had limited room for peripheral devices.  External keyboards and mice/trackballs were useful for reducing wear to laptop keypads and touch pads.  While no laptop experienced failure as result of water seepage into the keypad or touchpad areas, several laptops were observed with missing and/or sticking keys.  The cost to repair or replace a peripheral keypad or mouse/trackball is far less than to replace the laptop keypad or to replace the entire laptop.

The Auravision® EluminX^TM keyboard was deployed on several vessels and performed well. This keyboard was well-suited for the wheelhouse environment where lights are often kept low at night to reduce glare off the wheelhouse windows.  The keyboard is backlit, allowing for nighttime use without negatively impacting operator visibility.  A detracting feature of this keyboard is the PS/2 port connection to the laptop.  When connecting to newer laptops, a PS/2-to-USB adapter is required.  Trackballs seemed to be better suited to environments where computer space was limited, though many operators preferred typical computer mice.

Data Receipt, Processing, and Storage Systems

Data were exported off the vessel using the VMS and sent to the email address studyfleet@noaa.gov. Prior to exporting the data, the logbook software compressed the data files in a password-protected ZIP file. Additionally, files transmitted using the SkyMate® VMS needed to be converted from the binary ZIP format to a text format to an ACII text format through a UUencoding process. Once data were received at the Northeast Regional Office, a procedural language/structured query language (PLSQL) script extracted the data files from the emails, unzipped (and Uudecoded if necessary), archived, loaded to a series of raw Oracle tables, and lastly migrated the raw data to a set of formatted work tables. The raw tables were based on the same model as the work table model (Figure 4), though fields were not formatted in the raw table set. The raw tables served as a database archive of original data as submitted by the vessels, where as the work table set contained formatted data that would be subject to quality assurance/quality control (QA/QC) audits and available to the fishers to review and make changes. QA/QC controls were never implemented during the course of Phases I or II of the Pilot Program, nor was web-confirmation system ever built that would have allowed fishers access to their data. All data analyses covered in this report were conducted on data contained in the work table set.

DATA ANALYSIS

Overview

The data analyses presented in this report emphasize comparing the data collected by Study Fleet participants using the ELB systems to the two existing vessel-based fisheries-dependent data collection programs in the NMFS Northeast Region: FVTR and NEFOP.  Analyses were conducted for each level of data collection: trip, subtrip, catch (both kept and discard portions), and landings (i.e., dealer transactions).  These analyses (a) provide a review of the types of data collected by the program; (b) highlight areas where Study Fleet data could be used to complement existing data collection programs; and (c) highlight areas where Study Fleet data collection requires improvements.

The majority of the analyses utilized a triangulation approach to assess the relative accuracy of the data reported through the Study Fleet pilot program compared to existing fishery-dependent data collection programs.  When comparing the same observation collected via three different reporting systems, a better understanding of the benefits and shortcomings of the Study Fleet pilot program was attained.  For example: do self-reported haul-level data provide more precise and accurate data than do FVTR subtrip-level data?  Are self-reported haul-level data reliable in comparison to the haul-level data collected by trained at-sea observers (NEFOP data)?  All data collection methods have intrinsic measurement error whether caused by instrument or recorder error. Measurement error prohibits comparing the collected values to any known “true” value.  The term “misreporting” is used in these analyses to describe what are perceived to be incorrect values relative to the other data sources.  Usage of this term is not meant to imply that incorrect data were deliberately reported; there are many causes of misreporting, including unclear handwriting, incorrect coding, lack of training, and incomplete understanding of the regulations.

Data Sources

Study Fleet data were extracted from the NEFSC Study Fleet database (SFLEET).  Data collected with the Thistle Box were excluded from these analyses.  In order to protect vessel confidentiality, vessels fishing lobster pots during the course of the Study Fleet pilot program were excluded because there were fewer than three vessels using this gear.  The times series available for each vessel was contingent on when a particular vessel entered the Study Fleet pilot program.  The time series for an individual vessel started when the vessel sailed on the first trip reported in the Study Fleet database.  The first recorded Study Fleet trip began on September 8, 2003 and the last recorded trip landed on August 21, 2005.

FVTR data for each vessel (identified by permit number and/or hull identifier) were extracted from the NEFSC Vessel Trip Report database (VTR) for the corresponding time period when a vessel was participating in the Study Fleet pilot program.  Only FVTR data where data entry was complete and fishing occurred were used in the analyses.  NEFOP data for all Study Fleet vessels (identified by permit number and/or hull identifier) were extracted from the NEFSC observer database system (OBDBS).  All hauls were used regardless of whether the haul was observed for fish discards or not; of the total of the 3,119 NEFOP hauls extracted, 2,964 were observed (95.0%). This decision was made to retain the unobserved hauls because removal would reduce the sample size on which to run non discard comparisons (e.g., haul characteristics and retained catch analyses).  This effect would be cumulative for all FVTR vs. NEFOP comparisons because all NEFOP subtrips which include an unobserved haul would have to be removed from the analyses.  Inclusion of unobserved hauls impacts only the assessment of discarded catch reporting comparison involving NEFOP data at the trip-level.  In all instances, inclusion of these hauls could cause calculated NEFOP discard values to be biased low; however, given the low percentage of unobserved hauls included in the analyzed NEFOP data set, we did not feel there was sufficient cause to warrant their removal.

Landings-level matching was employed to match the dealer records from Study Fleet and FVTR data sets to federally permitted dealer weighout reports; dealer transactions are not recorded by the NEFOP, hence this data set was excluded from the analyses of landings.  Dealer weighout data were extracted from the NEFSC commercial fisheries database system (CFDBS). Dealer weighout data has traditionally been considered the most accurate estimate of landings of New England groundfish, thus, the dealer data set was treated as the benchmark data set to evaluate the accuracy of Study Fleet and FVTR landings data.

With the single exception of the data load delay analysis, all comparative analyses were restricted to matching vessels and matching trips among the various data collection programs: Study Fleet, FVTR, and NEFOP.  Before comparative analyses were performed, a matching procedure was conducted to establish comparison data sets.  These procedures ensured that information from the Study Fleet data set was properly matched with data in the comparative data set (e.g., catch records from a Study Fleet trip were being compared to the catch records observed on the same trip by a trained observer).  All matching procedures were programmed in SAS[6] V8.  The details of the matching procedure are described in the representative sections.

To determine whether systematic reporting bias existed between data collection programs (Study Fleet, FVTR, and NEFOP), a technique examining the distributions of differences was used.  Prior to testing for bias, the assumption of normality for the calculated differences was tested by using a Shapiro-Wilk test.  The vast majority of calculated differences violated normality assumptions (p < 0.0001).  A common test of paired observations, the paired t-test, is sensitive to normality thus the nonparametric Wilcoxon signed rank test was used (Zar 1999).  This is a test to determine if the median value of the distribution differs significantly from zero.

Trip-level Reporting

Many of the trip-level data elements are either mandatory across data sources, or fields used in the matching comparison, thus, a comparison of much of the trip-level information is not warranted (vessel, operator, port, crew numbers, trip type, etc.).  A general review of the Study Fleet data quality for these fields is covered in the data quality section.  It is noteworthy that for the time period examined, operator and FVTR serial number information were not collected by the NEFOP and neither the FVTR nor NEFOP programs captured the port of sailing.  While the home port of a vessel is captured by NEFOP, the home port of a vessel is not necessarily the port of sailing.

The trip matching procedure used a two tiered approach, with the first tier matching the vessel across data sets.  A match could exist on either the hull identifier or federal permit number since discrepancies were observed in these fields in at least one of the databases utilized.  The second tier required two of the following three data elements to match between data sets: sailing date; maximum landing date, or unique trip identifier (serial number from FVTR paper logbook).  The unique trip identifier could not be employed for matches between Study Fleet and NEFOP trips because the OBDBS does not contain this element.  This element was optional in Study Fleet.

Of the 1,108 trips recorded in the Study Fleet database (excluding trips using the Thistle box), a total of 641 trips (57.9%) from 23 unique vessels were matched to the FVTR database.  Fifty three trips (4.8%)[7] from 19 unique vessels were matched between the Study Fleet and the NEFOP database.  One hundred and sixty-eight trips from 27 unique vessels matched between FVTR and NEFOP database.  A total of 44 trips matched across all three databases.

Data Load Delay

To assess the timeliness of the data received from the three vessel-based data collection programs (Study Fleet, FVTR, NEFOP), the data load delays associated with each of the programs were compared.  Data load delay was defined as the number of days between the completion of a trip (date of landing) and the date when the data from that trip was available to data analysts.  Trips from all three programs with a date of landing between February 5, 2005 and May 31, 2005 were used regardless of whether the trips matched.  This time frame corresponds to the date when the Study Fleet database was installed at the NMFS NERO ( February 4, 2005), and trips were loaded to the database at regular intervals.  While Study Fleet data collection continued beyond May 31, 2005, regular database loads were no longer performed.  Inclusion of trips beyond this date would not provide an accurate picture of Study Fleet data timeliness.  Trips were binned into week intervals, and the weekly average delays were calculated.  The first week in the year is defined as that week containing the first Sunday of the year.

Over the examined period, Study Fleet data experienced a load delay of approximately 21.2 (± 7.2) days compared to 74.1 (± 19.0) days for FVTR and 187.9 (± 39.5) days for NEFOP data (Figure 5).  With the exception of FVTR data, fishing trips towards the end of the period experienced a shorter load delay compared to the start of the period.  Over the course of this period, the NEFOP was undergoing office relocation and transitioning through a period of high staff turnover (David Potter, pers. comm.).  A separate examination of the load delay experienced for 2004 NEFOP data suggested an average load delay of 88.5 (± 0.6) days and is consistent with the target data load time of 90 days (David Potter, December 12, 2005, pers. comm.).  Study Fleet load delay exhibited a decrease in the load delay over the period, primarily because of the increased usage of VMSs to transmit trip reports from sea.  Prior to use of VMS, data had to be manually collected by field technicians who would then email it to the Regional Office.  On February 5, 2005, 13 vessels were equipped with VMS units compared with 25 vessels on May 31, 2005.

During the period of time that the Study Fleet system was installed at the NERO and data were loaded on a regular basis, the smallest load delay observed was 0 days.  A 0-day load delay indicates that the trip report was loaded on the same day the vessel landed.  If a vessel was equipped with a VMS and trip reports were transmitted either shortly before or immediately after landing, the load delay would be contingent only on internal NMFS processing of the data.  During this period, the database load routines were run manually on a regular, but not daily, basis.  If the load routines were automated and scheduled to run at set intervals throughout the day, a 0-day delay would be reasonable for all trips.  Compared to other vessel-based data collection systems used in the Northeast Region, the Study Fleet data collection system represents a significant improvement in data timeliness. It is necessary to mention that both FVTR and NEFOP data undergo various levels of data auditing (during data entry, post-entry audits, etc.) prior to these data being made available to data analysts. The Study Fleet data examined were not put through any auditing procedures. They represent the data exactly as submitted by the vessel operator. However, the fact that Study Fleet data are unaudited is misleading; the electronic data collection process allows for auditing to occur when data are first entered by the operator, reducing the need for subsequent internal audits.

Haul and Subtrip-level Reporting

By using the matched trip data set, a haul-level matching procedure was developed to match individual hauls between the two data sets containing haul-level data, Study Fleet and NEFOP.  The date and time midpoints of all Study Fleet hauls associated with the trips contained in the Study Fleet - NEFOP matched trip set were calculated.  These hauls were then matched by finding all NEFOP hauls that had start and end date times that bracketed Study Fleet haul midpoints. A record count was performed on the unique identifiers of all haul records returned by the matching procedure, and only those hauls where a one-to-one match existed were retained. The resulting data set was then manually inspected for accuracy in the matching procedure; no matches were removed as result of the manual inspection. From the 53 Study Fleet-NEFOP matched trips, 355 individual efforts could be matched.

Because FVTR subtrips have no date/time component to them, Study Fleet and FVTR data sets had to be matched based on subtrip characteristics.  Individual Study Fleet hauls were “rolled-up” to the subtrip level by grouping on statistical area, gear type, and mesh size consistent with FVTR regulations.  Within subtrips, the numbers of hauls were summed, and the average gear quantity, gear size, soak duration and fishing depth per effort were calculated by using the arithmetic mean.  Subtrips were then matched across databases by using the matched trip data set, matching on statistical area, gear code, and mesh size.  From the 641 trips matched between Study Fleet and FVTR, there were 673 reported subtrips to the FVTR database and 743 calculated Study Fleet subtrips.  Assuming all statistical areas, gear types and mesh sizes were reported correctly to the Study Fleet pilot program, this suggests an underreporting of subtrips to the FVTR program.  There were a total of 438 subtrips between the two databases that could be matched. The 307 nonmatching Study Fleet subtrips could be due to underreporting of FVTR subtrips and/or incorrect reporting of statistical area, gear code, and mesh size to either the Study Fleet or FVTR programs.  We reduced the matching criteria element-by-element to determine the cause of the nonmatches (i.e., removed forced match on statistical area, gear code, mesh size, etc.).  When the various elements were removed from the match criteria the matching improved to 187, 305, and 155 unmatched subtrips for statistical area, gear type, and mesh size, respectively.  Matching of statistical areas and mesh sizes had the greatest impact on the number of unmatched subtrips.

NEFOP and FVTR data sets were matched at the subtrip level with a method similar to that used in the Study Fleet and FVTR match.  There were a total of 234 calculated NEFOP subtrips compared to 176 reported FVTR subtrips.  Like the Study Fleet - FVTR comparison, this discrepancy suggests that there is underreporting of subtrips in the FVTR data.  Using all matching criteria (statistical area, gear code, and mesh size) to match subtrips proved problematic.  When forcing a match on all three criteria, only subtrips where the longline gear was used were included in the results, a gear type for which mesh size is null.  The net mesh sizes reported by the two programs frequently differ likely because of differences in protocol and rounding effects resulting from the conversion of OBDBS mesh sizes to inches. The value recorded in the OBDBS database represents the inside mesh measurement as measured with calipers. It is either averages of multiple measurements (trawl codend and gillnet) or a randomly selected mesh (codend liner) measured in millimeters (trawl gear) or inches (gillnet) (cf. Observer Program Manual at http://www.nefsc.noaa.gov/fsb/).  The FVTR instructions (http://www.nero.noaa.gov/ro/fso/vtr_inst.pdf) request the measurement in inches for all gear types with no specification as to whether the mesh size measurement represents the inside or outside mesh measurement.  If a bag liner was present for NEFOP trawl gear subtrip, the mesh size of the codend liner was used as the mesh size measurement to ensure consistency with FVTR instructions which request either the mesh size of the codend or the mesh size of the liner.  To resolve the problems associated with matching on mesh size, a match was allowed if the FVTR mesh size value was within 10% of the NEFOP measured mesh size.  A record count was performed on the unique subtrip values of all subtrip records returned by the matching procedure, and only those subtrips where a one-to-one match existed were retained.  The resulting data set was then manually inspected to ensure one-to-one matches; no matches were removed as result of the manual inspection.  A total of 116 subtrips could be matched between the two data sets.

Gear Characteristics

All 355 matched Study Fleet vs. NEFOP hauls were associated with otter trawl gear.  The gear type matched exactly for all but eight hauls; in all eight hauls, the NEFOP database recorded midwater trawl gear while the Study Fleet database recorded fish bottom trawl gear.  Gear quantity matched for all 355 hauls (gear quantity = 1).  The gear size (footrope length) matched exactly for 222 hauls, while 325 hauls had gear size reported in the Study Fleet data set within 10% of the NEFOP data set.   Mesh size (codend mesh size or liner mesh size) did not match for any of hauls; for 176 hauls, Study Fleet values were within 10% of NEFOP values.  Mesh configuration (square or diamond) matched on 180 hauls.

Within the 438 matched Study Fleet vs. FVTR subtrips, there were 333 otter trawl subtrips, 15 shrimp trawl subtrips, 22 gillnet subtrips and 68 longline subtrips.  Because mesh size is a subtrip characteristic, comparisons of mesh size are uninformative between Study Fleet and FVTR.  The FVTR program does not collect mesh type information.  Gear quantity matched for 395 (90.2%) of the matched subtrips.  Longline gear quantity was most frequently reported differently between data sets, 22 out of 68 subtrips (32.4%), though by percentage, gillnet had the highest nonmatch rate at 40.9% (9 of 22 subtrips).  All other gear types had gear quantity matching rates greater than 95%.  The gear size matched exactly for 315 subtrips, while 400 subtrips had a gear size reported to SFLEET within 10% of the FVTRed value.  Among gear types, shrimp trawl had the highest (100%) nonmatch rate for gear size; however, all of the SFLEET values of the nonmatches were within 10% of the FVTR values.  Longline gear had the second highest nonmatch rate at 98.5% (68 of 69 subtrips).  Of the 68 nonmatched subtrips, 42 SFLEET subtrips had a gear size that was within 10% of the FVTR value.  All other gear types had gear size matching rates greater than 85%.

Of the 116 matched subtrips between NEFOP and FVTR, 37 were for otter trawl, 3 for shrimp trawl and 76 were longline subtrips.  There were no matched gillnet subtrips.  Gear quantity matched for 35 of the 37 otter trawl subtrips (94.6%); 100% of the shrimp trawl subtrips (3 of 3 records) matched, and none of the 76 longline subtrips matched.  The gear size matched for 16 of the 37 (56.8%) otter trawl subtrips, while 27 FVTR subtrips (73.0%) had gear size within 10% of the NEFOP value.  Only 1 of the 3 shrimp trawl subtrips matched on gear size; the gear size on the other two remaining FVTR subtrips exceeded 10% of the NEFOP recorded value.  None of the longline gear sizes matched, yet one of the 76 matched FVTR subtrips had a gear size within 10% of the NEFOP value.

The high percentage of nonmatching gear characteristics was unexpected, particularly between Study Fleet and FVTR given that Study Fleet participants were asked to follow the FVTR gear reporting protocol.  It was intended that fishers would enter all possible gear configurations (gear type, size, number, mesh size, and type) into the ELB prior to use and that while at sea they would select the appropriate gear configuration for the individual haul.  However, during port visits conducted during the voluntary portion of the project, many operators commented that they were unable to change their gear configurations during data entry for a particular haul. This indicates that the proper use of the ELB software was not effectively communicated to the fishers.  This issue may have lead to some of the discrepancies between Study Fleet and FVTR, particularly in the longline fishery where gear configurations are highly variable from trip to trip.  This is an issue that can be resolved through improved training of fishers on the proper use of the ELB.

Number of Hauls per Trip

The numbers of individual hauls recorded per trip were determined using trips identified by the trip-matching procedure for each of the pair-wise comparisons: Study Fleet vs. FVTR, Study Fleet vs. NEFOP, and NEFOP vs. FVTR.  An attempt was made to match at the subtrip-level for comparisons involving FVTR; however, misreporting of statistical area and mesh size on FVTRs made these comparisons difficult.  Comparisons were performed at the trip level by summing up the number of hauls per trip by gear type.  Differences in gear configuration (mesh size, mesh type, gear size, and gear quantity) were ignored.  The difference in the number of hauls between matched trips was then calculated for each of the possible pair-wise comparisons.  The distribution of differences was then plotted and examined for presence of directional bias (systematic over or underreporting of the number of hauls per trip).

Because of sample size limitations, the number of hauls per trip could be compared for only two gear types in the Study Fleet vs. NEFOP comparison: otter trawl and demersal longline trips.  There was only one matched gillnet trip with the Study Fleet recorded number of hauls four less than the NEFOP recorded hauls.  The median difference in the number of hauls recorded for otter trawl was 0.00; however, the Wilcoxon statistic was significant (n = 37, s = -24.0, p = 0.014).  This difference is likely because of the negatively-skewed distribution, where the number of Study Fleet hauls was less than the number of NEFOP hauls (Figure 6).  Study Fleet underestimation was more pronounced in the demersal longline comparison.  The median value was -5.00 and Wilcoxon statistic highly significant (n=13, s = -45.5, p < 0.001).

Similar patterns were observed in the Study Fleet vs. FVTR comparison.  All difference distributions were negatively skewed with the exception of shrimp otter trawl (Figure 7).  A negatively-skewed distribution suggests Study Fleet trip reports underestimated the number of hauls per trip.  Like the Study Fleet vs. NEFOP comparison, the median value for the Study Fleet vs. FVTR fish otter trawl comparison was 0.00 hauls, but the Wilcoxon statistic was again significant because of the negative skewness (n=448, s=-614.0, p=0.005).  The shrimp otter trawl comparison had a symmetrical distribution around the median value of 0.00 hauls, while the Wilcoxon statistic was not significant (n=36, s=-1.5, p=0.750).  Comparisons of  Study Fleet and FVTR estimates of number of hauls for demersal longline and sink gillnet suggest underestimation in the Study Fleet.  A median difference of -3.00 hauls was observed for demersal longline (n=68, s=-495.0, p<0.001) and -4.0 hauls for sink gillnet (n=81, s=-34.5, p<0.001).

The difference distributions of the number of hauls/trip for the NEFOP vs. FVTR comparisons were positively skewed suggesting an underestimation of number of hauls in FVTRs (Figure 8).  Both fish otter trawl and demersal longline had median values of 0.00 hauls, though the Wilcoxon statistic was not significant for otter trawl (n=37, s=-59.5, p=0.024).  The Wilcoxon statistic was significant for longline (n=76, s=237.5, p<0.001).

The underestimation of haul counts in Study Fleet data was expected.  It was known that many users experienced technical difficulties with the ELB, particularly earlier versions, which prohibited users from entering all hauls occurring on the trip.  Additionally, users occasionally forgot to enter hauls in the ELB.  Inspection of the fish otter trawl differences showed that the largest discrepancies for all Study Fleet comparisons were associated with trips where haul counts were large (>30 hauls/trip).  An additional problem affecting the recorded number of hauls for fixed gears was associated with miscommunication between project managers and fishers.  Early versions of the ELB software contained a “Number of Tows” field similar to the FVTR.  Many fixed gear fishers were combining all of their hauls into a single “subtrip” and filling out the ELB in a manner identical to a FVTR.  The “Number of Tows” field was exported from the software and included in the received data file; however, the final database tables did not contain this field because it was incorrectly assumed by program managers that this haul-based system would not contain combined hauls[8].  This problem was recognized late in the program (February 2005), and fixed-gear fishers were reluctant to change reporting practices stating that haul-level information was too burdensome given their fishing practices (i.e., hauling sets in quick succession).  Comparison of NEFOP and FVTR data suggests that FVTR data underestimated the number of hauls.  Careful inspection of those matched trips suggests that at least some of the discrepancies can be attributed to FVTR data entry errors, though fixed gear discrepancies may be associated with improper FVTRs reporting.

Haul Duration

Haul, or soak, duration comparisons between Study Fleet and NEFOP were matched at the haul-level by using the results of the haul-matching procedure.  Like the number of hauls/trip comparison, the difference distributions of the recorded haul duration were examined to determine the presence of directional reporting bias.  Only hauls from fish otter trawl could be matched.  The distribution of difference exhibited a negatively skewed bimodal distribution with modes at 0.00 and -8.00 minutes and overall median difference of -0.10 minutes (Figure 9). The median was statistically significant from zero (n = 355, s = -13823.5, p < 0.0001). 

Haul duration comparisons between Study Fleet and FVTR were performed at the subtrip-level by using the Study Fleet calculated average soak duration per haul.  There were four gear types among the matched subtrips.  The median fish otter trawl haul duration difference was 1.10 minutes (n = 332, s = 5824.2, p < 0.001) and exhibited a moderate positive skewness, indicating longer Study Fleet haul durations compared to FVTR estimates (Figure 10).  Shrimp otter trawl distributions were relatively symmetrical about zero with a median of 0.00 minutes (n = 15, s = -8.0, p = 0.607).  There was a high percentage of large negative differences (Study Fleet haul durations shorter than FVTR) in the demersal longline comparison (Figure 10).  While the median difference was 0.00, the Wilcoxon statistic was significant (n = 68, s = 19.5, p < 0.001) indicating Study Fleet haul duration estimates are different than FVTR estimates.  Sink gillnet duration differences were positively skewed with a median value of 11.00 minutes (n = 12, s = 19.5, p = 0.020).

NEFOP vs. FVTR haul duration comparisons were similar to those of Study Fleet vs. FVTR.  In this comparison, there were three gear types in the matched subtrips, the gillnet sample size (n = 3) was too small to provide meaningful results.  Fish otter trawl differences were positively skewed (Figure 11) with a median of 15.00 minutes; however, the Wilcoxon statistic was not significant (n = 37, s = 98, p = 0.141).  Demersal longline distributions were negatively skewed with median value of -84.14 minutes (n = 76, s = -1152, p < 0.001), indicating that reported FVTR haul durations were longer than those of NEFOP.

Study Fleet fish otter trawl haul durations were longer than those of FVTR, but shorter than NEFOP estimates.  For mobile gear such as otter trawls, the ELB automatically calculated haul duration based on the recorded start and stop times of the effort.  Start and stop times were established when the operator clicked on the associated buttons.  The Study Fleet protocol stated that the “Start Effort” button was to be pressed when the gear began fishing (i.e., winch break engaged) and the “Stop Effort” button be pressed when the gear ceased fishing (i.e., winch break released, retrieval of net commenced).  This procedure is different from the NEFOP protocol where start time corresponds to the moment when the net touches the water.  The high percentage (approx. 20%) of hauls with 0.00 minute differences suggests that either the Study Fleet or NEFOP protocols were not correctly followed. The lesser mode (-8.00 minute difference) can be partly explained by this difference in protocols as it takes several minutes to deploy gear, though the amount of time is variable by vessel, individual gear, and fishing depth (M. Palmer, unpublished data).

It is unclear why Study Fleet otter trawl haul duration estimates tend to overestimate effort compared to FVTRs.  One explanation is that the estimation of average of haul duration reported on FVTRs could be biased low (e.g., four hauls average 2 hours and 15 minutes and the operator reports 2 hours on the FVTR).  Another explanation could be that operators forget to press the “Stop Effort” button at the end of each effort when using the ELB.  A delay in the pressing the “Stop Effort” button could be responsible for the positively-skewed distributions observed in the Study Fleet vs. FVTR fish otter trawl effort durations differences (Figure 10).  Unlike Study Fleet and NEFOP duration estimates, FVTR haul durations constitute single point estimates of the average haul duration for all hauls within the subtrip.  To better understand the variability associated with FVTR point estimates, a cursory examination was performed on the distribution of haul durations from ten trips where more than 20 hauls/trip occurred. All of the FVTRs used in this examination reported average haul durations of 3 hours.  The distribution of individual haul durations as determined from Study Fleet data were plotted by using boxplots (Figure 12). There was a slight directional bias in the reported medians with six of the ten medians falling below the reported FVTR three hour average consistent with the first explanation. However, there was an equal number of means above and below the reported FVTR three hour average in addition to several large positive outliers likely caused by operators forgetting to press the “Stop Effort” button.

Fixed gear set duration comparisons suggest an over estimation of soak duration on FVTRs.  It was not possible to ground-truth either Study Fleet or NEFOP by comparing one another; however, both have very similar patterns when compared to FVTR demersal longline soak durations (Figure 10 and Figure 11).  For fixed gear, Study Fleet soak durations had to be manually entered into the ELB.  These soak duration estimates would not be susceptible to the same button activity that may have impacted Study Fleet mobile gear estimates.  Because of the consistency in results between Study Fleet and NEFOP compared to FVTR, it is likely that differences are the result of incorrect estimation when filling out FVTRs.

Statistical Area Reporting

The Study Fleet and the NEFOP collect higher resolution spatial data on fishing activity than does the FVTR program.  However, the FVTR protocols require that for each statistical area fished, a separate logbook sheet be filled; thus at the spatial scale of statistical area, all three programs should be equivalent if fishing activity is reported correctly.  Statistical areas from fishery-dependent data sources can be determined in two ways.  In FVTR and Study Fleet trip reports, both statistical areas and point locations (latitude and longitude or loran bearings) are reported.  Only the point locations associated with a haul are reported in NEFOP data.  The associated statistical area can be derived by using the given point locations.  The statistical area was derived from the point location corresponding to the end of the haul (when retrieval of gear commenced) for both Study Fleet and NEFOP data.  For FVTR data, only a single point location is provided per subtrip.  If both the provided and derived statistical areas were in agreement, then the provided area was used.  If the provided statistical area and the derived statistical area disagreed but were adjacent to one another, then the derived statistical area was used.  If the provided statistical area and the derived statistical area were not identical and were not adjacent, then it was assumed that the point location was unreliable and the statistical area was left blank.  If either the derived or provided statistical areas were null and the other was not, then the not null statistical area was used.  The number and identity of unique statistical areas fished per trip were then determined from all subtrips (FVTR) or hauls (Study Fleet and NEFOP).

Statistical areas fished were then compared across data sources to assess the level of agreement.  Trips were broken into two categories: single subtrip trips (fishing occurs in only one statistical area per trip) and multiple subtrip trips (fishing occurs in more than one statistical area per trip). Statistical areas were categorized as having been correctly reported if both the number of statistical areas reported and the identity of those statistical areas agreed.  For instance, if an observer recorded that fishing effort occurred in statistical areas 513 and 515 and the Study Fleet data also indicated that fishing occurred in 513 and 515, then the trip would be considered correctly reported trip.  If however, the Study Fleet data indicated that fishing occurred in only 513 or in areas 513 and 514, then these would be considered incorrectly reported Study Fleet trips.

From the 53 matched Study Fleet-NEFOP trips, there was an agreement in statistical area reporting for 90.6% of the trips.  Single-statistical area trips composed 66.0% (35 of 53 records) of the total matched trips; there was 100% agreement on all matched single-statistical area trips.  The remaining 34.0% (18 of 53 records) of the compared trips were multiple-statistical area trips, of which > 70% (13 of 18 records) of the trips agreed (Table 5).

There was a 73.6% (472 of 641 records) agreement in reporting of statistical areas among all 641 matched Study Fleet vs. FVTR trips.  Single-statistical area trips accounted for 77.2% (495 of 641 records) of all matched trips.  There was a 94.1% (466 of 495 records) agreement in reporting of single-statistical area trips.  Multiple-statistical area trips had an agreement rate of only 4.1% (6 of 145 records).  There was one trip among the 641 for which no statistical area(s) could be determined from either Study Fleet or FVTR data (Table 6).

To provide perspective to the Study Fleet and FVTR statistical area comparisons, FVTR statistical area reporting was compared to NEFOP statistical area reporting.  Reporting trends were similar to those observed between Study Fleet and FVTR.  There were a total 168 matched trips, of which 70.8% (119 of 168 records) were in agreement on the reporting of statistical areas. Single-statistical area trips accounted for 74.4% (125 of 168 records), of which 94.4% (118 of 125 records) were in agreement.  Multiple statistical area trips accounted for 47.6% (40 of 168 records) of all matched trips, with 2.5% (1 of 40) of the trips being in agreement.  There were three trips from which the FVTR statistical area(s) could not be determined (Table 7).

To assess the impact of statistical area misreporting, NEFOP trips between 1989 and 2005 were used to determine the percentage of groundfish landings that could potentially be affected by statistical area misreporting.  Standard sea sampling trips with reported groundfish landings were extracted from the NEFOP database.  Groundfish catch (kept and discard) was then converted to live weight in pounds and summed by trip.  Trips were divided into two categories: single subtrip trips (fishing occurred in only a single statistical area) and multiple subtrip trips (fishing occurred in more than one statistical area).  The groundfish landings and the number of trips were then summarized by year and trip category, and percent compositions of each category were calculated.  Results confirm the previous assumption; while multiple statistical area trips may constitute only 10-30% of the overall trips, they are responsible for a disproportionate percentage (in some years > 50%) of the overall groundfish catch (Figure 13).

Many of the New England groundfish species are assessed by stock (Gulf of Maine cod, Georges Bank cod, etc.) whose boundaries are defined by statistical areas.  Commercial landings for specific fish stocks are determined in part by the statistical areas reported on FVTRs (Wigley et al. 1998).  Correct reporting of statistical areas in self-reported data is necessary to accurately estimate fishing mortality.  To assess the accuracy of statistical area reporting in Study Fleet data, the number of statistical areas reported per trip and the degree of matching were compared among all matched trips across all data sources.  These results indicate that positional information, at the statistical area level, obtained from Study Fleet data is reasonably close to what is obtained from NEFOP information and a significant improvement over area fished reported on FVTRs, most notably for multiple statistical area trips.  While the majority of trips fish in a single statistical area, the multiple statistical area trips account for a disproportionate percentage of the landings because typically these trips are from the larger, higher-capacity offshore vessels.  It is recognized that misreporting of statistical area does not necessarily translate to misallocation of stock landings (i.e., so long as the removal is attributed to a statistical area associated with the same stock complex).  However, it is important to consider that use of FVTR point locations for finer-scale analyses (e.g., Murray 2005) are subject to the same types of uncertainty associated with statistical area reporting.

Catch-level Reporting

Catch-level matching was performed at both the trip level and haul level.  Prior to the implementation of either of the matching procedures, all species catch (both kept and discarded portions) records from each of the databases were converted from reported quantities to live (round) pounds by using standard Northeast Region Commercial Fisheries Database System (CFDBS) conversion factors for the species.  Study Fleet data were converted to live pounds based on the recorded species code, market code, grade code, and unit of measure.  FVTR catch records were converted BY using the VTR species code.  All VTR species codes are assumed to be of an “unknown” market category, “round” grade category and units of measure in pounds unless the code implies differently (e.g., “SCALB”, sea scallops (Placopecten magellanicus) reported in bushels, “MONKT”, monkfish (Lophius americanus) tails.).  NEFOP species weights are reported as either “round” or “dressed” weights; all weights were converted to round live weight.

Because of a lack of species-level reporting of the seven skate species (Rajidae) in self-reported (Study Fleet and FVTR) data, all skate species were collectively grouped as “skates”.  Prior to matching, the procedure summed all species catch records from individual trips, grouping by species and disposition (kept or discarded).  The trip-level catch matching procedure then matched trip-catch records from those trips identified in the trip-matching procedure, by using the trip identifier, species, and disposition to join the data sets.  The two data sets were matched such that all records from each of the data sets were returned even if records did not satisfy the join condition. This type of match (i.e., database outer join) facilitates comparison of matched records such that the amount of nonmatching can be quantified.

The effort-level matching procedure summed all species catch records from individual hauls identified by the effort-level matching step, grouped them by species and disposition (kept or discarded), and then matched effort-catch records by using a procedure to identify matches and nonmatches at a trip, haul, species, and disposition level.

The first catch-record analysis compared the reporting frequency of trip-level matching catch records (i.e., how often species were reported in Study Fleet data but not in FVTR, and vice versa, etc.).  Analyses were performed separately for the kept and discarded portions.  Catch reporting was compared between Study Fleet and FVTR, Study Fleet and NEFOP, and FVTR and NEFOP.  Kept and discarded reporting was divided into three categories: (1) not reported to database X, but reported to database Y; (2) reported to both database X and Y; (3) not reported to database Y, but reported to database X.  Species in categories 1 or 2 were grouped by species and frequency counts performed to determine if there were systematic omissions of particular species from each of the data collection programs.

The second catch analysis used the catch that could be matched between data sources (category 2 records from above).  Matching catch was examined for the presence of reporting bias (systematic under- or over-reporting of catch amounts). During the course of the Study Fleet pilot program, data quality checks revealed that some fishers were inadvertently reporting whole monkfish as monkfish tails when reporting the discarded portion of their catch.  Study Fleet estimates of monkfish catches were consistently 3.32 times the live weight in pounds of monkfish reported to the other two databases; 3.32 is the CFDBS conversion factor applied to monkfish tails.  Rather than infer the reporting intent of fishers, monkfish were removed from all discard bias analyses. Additionally, based on port interviews of Study Fleet participants, it was known that trip catch summary reports generated by the ELB software were often used to fill out the FVTR paper logbooks resulting in identical reporting patterns across data sources.  To improve the likelihood that compared catch records were independent estimates, all catch records where the values were exact between all sources were removed. In all comparisons more than 80% of the original data sets remained to analyze after removal of identical catch quantities.

Prior to testing for bias, the assumption of normality for the calculated differences was tested with a Shapiro-Wilk test.  In all instances, the null hypothesis of normality was rejected (p < 0.0001).  The Wilcoxon signed-rank test was used to test for reporting bias.  Equivalent bias analyses were performed on catch that could be matched at the haul-level between the two haul-level programs, Study Fleet and NEFOP.

An additional comparative analysis was performed for all catch where a match could be established across all three data sources.  The matched catches were compared by using ternary diagrams.  Ternary diagrams provide a visual tool to assess relative distributions when three properties are being explored, in this case, comparing catch amount reporting between Study Fleet, NEFOP, and FVTR data sources.  As in the pair-wise comparisons, records where the reported catch poundage was equal across all three data sources were excluded from these analyses. This resulted in the truncation of the kept-catch data set from n=171 to n=118 and the truncation of the discarded-data set from n=82 to n=19.  Reported amounts were normalized so the plotted values represent percents of the total (the reported quantity for each data source represents a percentage of the sum of the three, or a percent composition); for example: 250 lb. of Atlantic cod (Gadus morhua) is reported to Study Fleet, 300 lb. to FVTR, and 350 lb. are reported in the NEFOP database.  Ternary normalization would result in the following values: Study Fleet 27.8%; FVTR 33.3%; and, NEFOP 38.9%.

Retained (Kept) Catch Reporting

There were 3,959 different kept species entries/records from matched Study Fleet and FVTR trips. Study Fleet and FVTR species kept records matched for 80.2% of the total records (i.e., both Study Fleet and FVTR data indicated that the individual species was reported as kept catch). On 9.6% of the total kept species records, Study Fleet data reported the species as retained kept catch, but a matching species record could not be found in FVTR data.  Conversely, 10.6% of the kept species records were reported on the FVTR but not reported in Study Fleet data (Figure 14).  A total of 362 different kept species records existed for matched Study Fleet and NEFOP trips.  Study Fleet and NEFOP data reported the same kept species for 69.3% of the records, while only 26.2% of the total records were recorded as kept in the NEFOP data but not reported in Study Fleet data; and 4.4% of the records were reported in Study Fleet data but not recorded in NEFOP data.  Comparison of the 1,083 different species kept records kept from matching FVTR and NEFOP trips, revealed a 66.4% agreement.  Kept records reported in NEFOP data but not found in FVTR data composed 26.0% of the total records, 7.6% of the total species-kept records were reported in FVTR data but not recorded in NEFOP data.

Examination of those species most frequently not reported to the three data sources did not reveal systematic nonreporting of species, with the exception of American lobster (Homarus americanus) (Table 8). Lobster went unreported in self-reported data (Study Fleet and FVTR) 54.0 – 94.7% of the time. Study Fleet data had the lowest lobster reporting rate.  The omission of lobster may be partly explained by the fact that lobster is often retained for home consumption, and fishers may not realize that species retained for home consumption are required to be reported; however, a separate analysis of species reported as home consumption in 2004-2005 FVTR did not find lobster to be in the list of the top 10 species most frequently utilized for home consumption (M. Palmer, unpublished data). Another explanation is that since lobster are caught less frequently than other retained species, they are more likely to go unnoticed by the captain when catch is reported (i.e., pulled from the pile and placed in holding tanks before the captain has a chance to estimate the catch).

In general, the percent omission was low (< 30%) for all other species in the Study Fleet/FVTR comparisons, though a high omission rate was also observed for weakfish (Cynoscion regalis) records.  Overall, individual species omission rates between Study Fleet and NEFOP were relatively low; however, because of the small sample size, (< 30 records per species), it is difficult to draw conclusions from these data as accidental omissions of a few records could artificially inflate omission rates.  The FVTR vs. NEFOP comparisons suggest high FVTR omission rates for many species including sea scallop, white hake (Urophycis tenuis) and Atlantic halibut (Hippoglossus hippoglossus).  Atlantic wolffish (Anarhichas lupus) had a high omission rate in NEFOP data as evidenced in both the FVTR vs. NEFOP and Study Fleet vs. NEFOP comparisons.  The red hake (Urophycis chuss) and white hake complex (two species reported together) is not reported in NEFOP data because sampling protocol requires reporting to the species level; these species would have been reported as red hake and/or white hake in NEFOP data.

Comparison of the kept weight/amounts reported for those kept records that could be matched across data sources suggests that Study Fleet kept estimates are less than those of both FVTR and NEFOP (Figure 15).  Median differences were negative (p < 0.001) for both Study Fleet comparisons of reported weight, but more so when compared to NEFOP weight (Table 9).  NEFOP reported amounts were generally greater than FVTR amounts, with a positive median difference (p < 0.001) observed (difference = NEFOP - FVTR).  Ternary diagram of the reported kept amounts supports these results, with reported amounts tending to be greatest in the NEFOP data (evidenced by number of points in the NEFOP realm), followed by FVTR and Study Fleet (Figure 16).  Comparison of species kept records matched at the haul level between Study Fleet and NEFOP also indicates that Study Fleet estimates tend to be lower than NEFOP estimates with differences negatively skewed (Figure 17).

Discarded Catch Reporting

A total of 3,243 unique discarded species records existed between match Study Fleet and FVTR trips.  Of that total, 43.8% discarded species records were reported to both Study Fleet and FVTR programs, 44.8% reported only to Study Fleet, and 11.4% reported only to the FVTR program.  There were 629 unique discarded species records between Study Fleet and NEFOP, with 31.6% of the records were reported to both data sources, 62.2% of the records were recorded in NEFOP data but not in Study Fleet data, while only 6.2% reported to the Study Fleet pilot program but not recorded in NEFOP data.  Between NEFOP and FVTR, 15.7% of the 1,661 discarded species records existed in both data sources, 82.4% were recorded by the NEFOP but not reported in FVTR data, while only 1.9% of the total were reported in FVTR data but not recorded by NEFOP (Figure 14).

Most species omitted from Study Fleet discarded records had omission rates below 10% (Table 10), with the exception of American lobster (90.8%), pollock (Pollachius virens) (62.5%), white hake (44.4%), and Atlantic cod (15.5%).   FVTR omission rates were generally high (>30%) for the top ten omitted species by record count.  At the species level, there was a higher omission rate of many species when comparing FVTR to Study Fleet data.  These species included yellowtail flounder (Limanda ferruginea), winter flounder (Pseudopleuronectes americanus), monkfish, skates, spiny dogfish (Squalus acanthias).  When comparing species omission of Study Fleet to NEFOP, lobster again was the most frequently omitted species (95.7%), followed sea raven (Hemitripterus americanus) and starfish (Asteroidea).  While the percentage of some NEFOP omitted species was high (> 50% in some cases), the records counts were all low (≤ 6).  The sculpin code used in the self-reported fishery data collection systems refers to sculpin unclassified (Cottidae); observers are more likely to identify the particular sculpin to species, in these instances most likely the longhorn sculpin (Myoxocephalus octodecimspinosus).  When comparing FVTR to NEFOP, sea raven and starfish had 100% omission with many other noncommercial species exhibiting very high percent omission (> 75%).  The 100% NEFOP omission for the red/white hake complex is expected because fishers are allowed to report the aggregate hake complex, but observers record hake to the species level.  Sculpin also exhibited an elevated omission percentage (27.3%) which is likely due to the species-level reporting issue mentioned previously.

NEFOP discard weight estimates tended to be higher than self-reported (Study Fleet and FVTR) discarded amounts when compared at the trip-level, though less so than kept amounts (Table 9, Figure 15).  The difference was greatest between NEFOP and FVTR.  While the distribution of differences between Study Fleet and FVTR  discarded pounds appear negatively skewed, indicating higher Study Fleet discard estimates, the median value was 0.00 and the Wilcoxon statistic was not significant (p = 0.606).  Ternary diagram of the discarded amounts supports these results, with amounts tending to be greatest in the NEFOP data, followed by Study Fleet and FVTR respectively (Figure 16).  Comparison of Study Fleet and NEFOP discarded amounts at the haul-level showed no significant difference (p = 0.720, Table 9); the distribution of differences is relatively uniform about 0.00 (Figure 17). This suggests that there may be some minor, but not statistically significant, bias in haul-level discard estimates that is increased to a significant level when summed across all hauls for a trip.

Study Fleet discarded catch reporting was slightly improved over that of FVTR.  While omission rates were high (> 60%), this represented a 25% decrease in omission rates compared to FVTR (> 80%).  Omission rates for several of the heavily exploited species (yellowtail flounder, winter flounder, monkfish, spiny dogfish, etc.) were substantially improved in Study Fleet reporting (Table 10).  Study Fleet reporting of discarded amounts was slightly better than reporting of the kept catch reporting in that there was no observable bias compared to FVTR subtrip-level reporting or NEFOP haul-level reporting. It is possible that small, undetectable, biases at the haul-level may lead to larger biases when summed over the entire trip.  There is evidence of this possibility in the reporting of discarded catch within the Study Fleet. However, the median difference is smaller between Study Fleet and NEFOP than with FVTR (Table 9).  Similar to kept-catch reporting, training should focus on improving captain hail weights of the discarded catch and stressing the importance of reporting all discarded species.

Landings-level Analyses

Landings-level matching was employed to match the dealer records from Study Fleet and FVTR data sets to federally permitted dealer weighout reports; dealer transactions are not recorded by the NEFOP, hence this data set was excluded from this analysis. Dealer weighout data have traditionally been considered the most accurate estimate of landings of New England groundfish, thus, the dealer data set was treated as the benchmark data set to evaluate the accuracy of Study Fleet and FVTR landings data.

Before initiating the matching procedure, all landings records associated with those trips identified by the trip-matching procedure were extracted from the two vessel-based databases, Study Fleet and FVTR; landings records with a null dealer number were excluded.  Like the catch records, landings records were first converted to live pounds, skate species aggregated to the generic “skate” category and record counts and weights were summed by trip, species, dealer, and date sold prior to matching.  The two data sets were matched by trip, species, dealer’s federal permit number, and/or the date sold.  If the matching procedure could not match on both dealer permit number and date sold, it attempted to match on only dealer permit number.  If a match could not be attained, a match was attempted on the date sold field.

All 2004 vs. 2005 dealer data with vessel permit number corresponding to Study Fleet vessels were extracted from the dealer weighout database, CFDBS.  The matching results from the vessel-based data were then matched to dealer weighout data such that the final data set contained all of the records from the Study Fleet-FVTR data set, but only the matching records from the dealer weighout data set (i.e., left database join).  While this type of join artificially reduces the number of dealer records and can bias our results, this was necessary to reduce the number of false positive nonmatches. Even though the dealer weighout database contains a field to record the unique trip identifier, it was frequently missing in the data; it was difficult to establish matching trips from the dealer weighout database since this database does not record the begin and end dates of individual fishing trips.

The landings match employs vessel permit number and hull identifier, species, and the dealer’s federal permit number or date sold from either the Study Fleet or FVTR data sources.  The dealer number and date sold values used in the match did not have to come from the same data source (i.e., the match was allowed to use the dealer permit number from Study Fleet and the date sold from FVTR, or vice versa), but both the dealer number and the date purchased/sold had to have a match.  This was done to allow matches in instances where transcriptions errors on one of the matching fields, either the part of the vessel operator or data entry staff, would have prohibited the establishment of a match.

The results of the matched landings records were grouped into four major categories. Within each of the major categories there are three or four subcategories.  The record counts and percent of the total matched set were calculated for each of the subcategories. Each category was further broken into subcategories which are explained in Table 11.

1. Dealer transaction records could be matched across all three databases: records where either the dealer number or date sold could be matched among all databases.

2. Dealer transactions records from only two of the three databases present: records where either the dealer number or date sold could be matched among two of the three databases.

3. Nondealer transactions (dealer codes 99998, 00001, 00002, 00003, 00004): records reported to Study Fleet or FVTR where the landings were not sold to a federally permitted dealer (kept as bait, home consumption, etc.).  These are records for which a dealer weighout match is not expected.

4. No confirmation possible because of insufficient match of dealer transaction records: records where no match could be established between any of the databases.

Landings records that could be matched across databases (categories 1 and 2 above) were examined for the presence of reporting bias (systematic under- or overreporting).  Like the catch comparisons, records where the database values were exact were removed from the bias analyses.  Landings records that could be matched across all three databases were examined by using ternary plots after normalizing the reported quantities/kept weight.

Landings Reporting

Of the total 4,129 records returned by the landings matching procedure, 2,691 (65.2%) could be matched across all three databases, with dealer number agreement slightly better in Study Fleet data compared to FVTR data (category 1, Table 11).  Approximately 10.2% of the nonmatching records suggest a compliance issue; either the purchase was not reported by the dealer (subcategory 2a, 6.2%) or the landing record was not reported on the vessel’s FVTR (subcategory 2b, 4.0%).  Nearly 10% of the matched landings records had no Study Fleet match (subcategories 2c, 2d).  It is likely that this was caused by either the use of invalid species codes or nonentry of landings information, two data quality issues that are covered in the data quality section.  There was a high incidence of omission of nondealer transactions from Study Fleet data (subcategory 3b, 105 of 111 nondealer records).  For 9.6% of the records no comparison could be made because of insufficient matching across data sources (category 4).

Bias tests on the matched landings records suggest that the Study Fleet landings estimates (weight in pounds) tend to be lower compared to FVTR landings estimates (Table 12), though the difference distribution is relatively uniform (Figure  18).  Comparison of both self-reported landings weight to dealer data indicated that self-reported data tends to underestimate actual landings amounts.  These results are supported by ternary plot of the matched (n=1,283) landings records (Figure 19) where the majority of observations fell within the dealer region compared to either FVTR or Study Fleet, and slightly more points exist within the Study Fleet region compared to FVTR.

Absent the omission of Study Fleet landings records, which are discussed separately in the data quality section, results from the analyses of landings data suggest that entry of landings information into an ELB, as was the case in the Study Fleet pilot program, may enhance the quality of data received (i.e., correct recording of dealer permit numbers) because Study Fleet data processing is not subject to interpretation of handwriting or potential transcription errors on the part of data entry staff.  Nonreporting of home consumption and catch retained as bait (i.e., the nondealer codes) is a significant problem in Study Fleet data.  There were 105 records (2.5%) reported as nondealer landings that went unreported in the Study Fleet database.  The amount of nonreporting of nondealer transactions in the FVTR data set that is indicated by landings analysis is quite low (0.1%), however, as observed in the nonreporting of kept catch, the omission of lobster records in all self-reported data seemed to indicate low reporting of nondealer transactions in both Study Fleet and FVTR.  Because Study Fleet landings records are a summation of haul-based estimates, the accumulated error in haul hail weights could lead to a large discrepancy in the estimated landings based on the summation of individual haul hail weights. There is evidence of this in the Study Fleet vs. FVTR landings comparison (Figure 18a, Table 12), which is consistent with the kept catch comparisons (Table 9). However, the comparison of Study Fleet and FVTR landings with dealer weighout data suggests FVTR landings estimates tend to be less (as evidenced by the median values), a finding supported by the ternary plot.  Given the conflicting evidence, the concern of Study Fleet landings estimates falling below FVTR estimates because of the haul-level estimation is warranted.  This concern was addressed in the design of the logbook by allowing landings estimates to be estimated independently of the summation of the kept portion of the catch.  This would allow operators to correct the total landed amount at the end of the trip without impacting the individual catch estimates.  It was the intent of the Study Fleet pilot program that each independent hail weight represented a “good faith” attempt on the part of the captain to estimate catch or landings, thus errors in hails weight of individual catches would not impact the hail weights of total landings.

STUDY FLEET DATA QUALITY REVIEW

A general examination of data quality was performed on all Study Fleet data.  Because many data quality issues associated with reported quantities were addressed in prior analyses, these examinations focused on omitted data on a field-by-field basis.  Each of the individual record types (trip, effort, catch, and landing) were examined separately and categorized as null or not null. The null records were summed by logbook type and version number for each record type, trip, haul, catch, and landing (Table 13, Table 14, Table 15, and Table 16).  There is a general trend of decreasing occurrence (as measured by percent of total records) of null fields through subsequent logbook versions resulting from a combination of increased quality checks in the software and improved training for the fishers.  The overall data quality of the PSW software is better than that of the UNH logbook; however, PSW development benefited from many of the lessons learned from earlier UNH logbook versions and the limited deployment on two vessels; it was difficult to differentiate vessel effects (some operators were more careful with data entry) from software effects.  There are some data quality issues that require additional explanation.

In all data quality summary tables, the number of UNH logbook trips generally increases with the version number (Table 13, Table 14, Table 15, and Table 16).  This increase roughly coincides with the increase in the number of participants from Phase I to Phase II, but also with the duration of deployment with v4.0 was used for a longer period of time than earlier versions.  The PSW logbook v7.08 was only deployed in Phase II, and v7.09 and v7.10 were deployed very briefly in the final months of the project (Table 2).

Two of the fields in the trip record were not required by any of the software versions: operator permit number and FVTR serial number.  Because logbook reporting did not fulfill a vessel’s legal reporting requirements, these two fields were never made mandatory.  As result, this information was often not provided (Table 13).  There was a push in Phase II to increase the reporting of the FVTR serial number through improved training.  It was hoped that this would assist in process of matching Study Fleet data to FVTR data.  Fishers were asked to report the serial number printed on the FVTR logbook sheet used on the trip.  As with FVTR requirements, in the case where the trip involved multiple subtrips and more than one logbook sheet needed to be used, fishers were asked to report just one of the serial numbers.  A FVTR serial number was reported on approximately 30% of the UNH v4.0 trips and 75% of PSW trips.  The large percentage of null ports, both sailing and landing, are due predominantly to incompatibilities between the port support table used in the UNH logbook software and the Regional Office’s master database support table.  The UNH logbook used an older internal NMFS coding system for ports whereas the in-house database used the Federal Information Processing Standards (FIPS) coding system.  Not all of the internal NMFS codes have FIPS equivalents.  In earlier versions of the UNH logbook trip end date and sailing port were not collected, and there were no quality controls on the port of landing.

Among haul records, the majority of fields in the Phase II software versions (UNH v3.0 and v4.0 and all PSW versions) were populated (Table 14).  The exceptions to this were mesh size, mesh type, and average temperature. Mesh type was only a required field for trawl gear; for all nontrawl gear this field should have been null.  The temperature field would have only been populated if the vessel was equipped with a temperature probe.  The temperature probe received only very limited use on board three vessels.

The two largest data quality issues in the catch records were null species codes and nonstandard species codes (Table 15).  The predominant increase in the number of null species in v4.0 was due to internal miscommunication.  There was a desire on the part of the fishers to have a species code added to the UNH logbook that allowed them to report lobsters in numbers.  To accommodate this, the code “LOBC” (American lobster, count) was added to the software species support table.  This change was never coordinated with the database manager of the Study Fleet database and as such, the code was never added to the master species table that is used to validate all incoming species codes.  As a result, all catch records using “LOBC” code lost the species information when the records were loaded into the Study Fleet database.  The “LOBC” code is a nonstandard code that has no conversion to either the NMFS internal species coding system or the species coding system used in the Study Fleet database which is based on the codes used by the Integrated Taxonomic Information System (ITIS).

The occurrence of nonstandard species codes is indicative of another internal miscommunication in logbook design where the species code and units of measure were not coupled.  The ELB give the fishers the flexibility to report the variety of species caught and, for some species, also the grade (tails, livers, wings, etc.) and the unit of measure (pounds, bushels, gallons, trays, etc.); however, not all combinations of species codes, grade, and unit of measures are valid.  By restricting species codes, grades, and unit of measure to only the valid combinations, it is anticipated that this enhancement will eliminate many of the nonstandard species codes.  Although this enhancement was incorporated in v4.2, this software version has not been released.  

Many of the data quality problems in landings records are results of issues that affected the other record types, such as problems with species codes and ports (Table 16).  All versions of the UNH logbook have very few quality controls on the dealer data elements.  For example, dealer name and federal permit number were free-typed by the operator, and no checks existed to confirm whether this information is entered by fishers.  In later versions of the software, the reporting rate was improved primarily through training.  These shortcomings were improved in the unreleased v4.2.

FLEET COMPOSITION

The primary goal of the Study Fleet pilot program was to assemble a group of commercial vessels capable of providing high resolution fisheries data from the New England groundfish industry representative of “normal” fishing operations.  Multiple fleet deployment strategies were discussed. One method would seek to target high-capacity vessels.  This method would allow collection of detailed catch and effort data on a majority proportion of the total aggregate landings (> 70%) by covering only a relatively small percentage (< 20%) of the entire groundfish fleet (Figure 20).  A second method would stratify the New England groundfish fleet by its various properties (vessel size, gear types, ports, target species, areas fished, etc.) and attempt to achieve a representative sample from each stratum, or from those strata designated as “high priority” and in need of additional sampling.

The assembled Study Fleet did not fully realize either of these deployment strategies, but rather recognized the need to construct an initial fleet that would be conducive to the pilot nature of this project and to the iterative and demanding collaboration required to develop an operational software system.  The primary focus of the pilot program was to develop and test the managerial and technological processes.  Developing robust electronic reporting technology necessitated that the technology be tested under the various fishing conditions typical of the range of vessels that compose the New England groundfish fleet, including: (1) mobile and fixed gear, (2) closed and open wheelhouse, and (3) large and small vessel crews.  It was logical to concentrate the fleets around the geographical areas covered by the representative NGOs to facilitate coordination, training, hardware deployment, software updates, and technical troubleshooting.

The fleet was divided into three geographic sectors:   Gulf of Maine, outer Cape Cod, and southern New England.  The management of each geographic sector was subcontracted to three NGOs: Gulf of Maine Research Institute, Cape Cod Commercial Hook Fishermen’s Association, and the Manomet Center for Conservation Sciences.  The general characteristics of each geographic sector are highlighted in Table 17.  In Phase I, each of the organizations managed a fleet of five participants.  In Phase II, the fleet size of each NGO was expanded to 10 participants, bringing the total number of Study Fleet participants to 30.  Because some participants fished multiple vessels, the number of vessels managed by each NGO was not always equal to the number of participants.

Composition of the Study Fleet Compared to the New England Groundfish fleet

To compare the Study Fleet to the first fleet deployment strategy presented above (targeting high-capacity vessels), the 2004 landings were extracted from the dealer weighout database for all New England landings and summed by vessel permit number.  New England landings were defined as all landings attributed to a port in Maine, New Hampshire, Massachusetts, and Rhode Island.  Landings not attributed to an individual vessel (permits equal to 000000, 190998, or 390998) were omitted.  The vessel landings were then ranked in descending order, and the cumulative percent landings were plotted as function of cumulative percent vessels (Figure 20); high-capacity vessels fall on the left side of the plotted distribution. The distribution was recreated by using only groundfish landings.  Groundfish were defined as the 12 species managed under the Northeast Multispecies Fishery Management Plan:  Atlantic cod, haddock (Melanogrammus aeglefinus), pollock, white hake, Acadian redfish (Sebastes fasciatus), ocean pout (Zoarces americanus), yellowtail flounder, winter flounder, witch flounder (Glyptocephalus cynoglossus), windowpane flounder (Scophthalmus aquosus), American plaice (Hippoglossoides platessoides), and Atlantic halibut.  For both fleets, Study Fleet vessels are highlighted to provide a comparison of the assembled Study Fleet to the New England groundfish fleet and fleet as a whole.

Study Fleet coverage was more representative of the groundfish fleet relative to the entire New England (NE) fleet.  Only seven Study Fleet vessels were within the 20^th percentile of vessels responsible for total landings compared to 20 vessels within the 20^th percentile of vessels responsible for groundfish landings.  When considering just the NE groundfish fleet, the Study Fleet composition is a mixture of the two deployment strategies.  The Study Fleet covers the entire range of the spectrum, from high capacity vessels (i.e., those in the lower 20%) to the low capacity vessels (part-time vessels) with the majority of the fleet falling within the 60th – 85th percentiles.  

To compare the Study Fleet to the second deployment strategy (stratified coverage) the 2004 landings data were extracted from the dealer weighout database in a similar manner as described above and were divided into four groups for comparative purposes: the entire New England fleet landings, groundfish fleet landings, landings from groundfish trips only, and landings by Study Fleet vessels.  The groundfish fleet includes all vessels taking at least one groundfish trip in 2004.  Groundfish trips were defined as any trip where the cumulative sum of groundfish landings exceeded all other species (> 50%).  Fleet characteristics were compared for the following strata: ton class, gear type, port, and species landings.  The gear analysis did not include any data where gear was unknown.  In all cases, sensitive fisheries-dependent data presented adhere to the “rule of three” to protect vessel confidentiality.  Aggregated data pertaining to catch, landings, or fishing location are not disclosed unless the aggregation includes data from at least three individual vessels. For area fished comparisons, FVTR data were used in place of dealer data[9].

Overall, the Study Fleet composition was similar to that of the groundfish fleet; however, it was not consistent with the composition of the NE fleet as a whole or with the fleet that participates in groundfish specific trips.  It is important to note that the collected Study Fleet data include all trips taken by these vessels, not only groundfish trips.  The vessel size distribution of the Study Fleet (Figure  21) exhibits a similar distribution as seen in the cumulative groundfish landings distributions (Figure 20), where Study Fleet underrepresents the small and large capacity vessels and over represents the medium capacity vessels (ton class 3).  Of the four major New England fishing ports utilized by the groundfish fleet, the Study Fleet underrepresented New Bedford, MA, and Gloucester, MA, and closely matched Point Judith, RI, and Portland, ME, based on the total percentage of reported 2004 landings (Table 18).  The underrepresentation of the top two ports was caused by overrepresentation of some of the minor ports such as Chatham, MA, Newport, RI, and Harwich Port, MA.  Of the five major gear types, trawl gear was slightly underrepresented and longline gear over-represented (Table 19).  Differences in gillnet categories are due to reporting resolution in gear types; combining the “gill net, other” and “sink gillnet” categories achieves approximately equal representative coverage for all fleet sectors with the exception of the entire NE fleet.  Among landed species, haddock represent a major discrepancy between the Study Fleet and the groundfish fleet and NE fleet (Table 20).  Two of the more predominant NE fleet landed species, Atlantic herring (Clupea harengus) and sea scallops, received negligible coverage by the Study Fleet.

Fishing effort captured by the Study Fleet pilot program ranged from the mid-Atlantic (statistical areas 611 - 629) to the Gulf of Maine (statistical areas 511 - 515), though more than 95% of the effort occurred in the New England region (statistical areas < 600).  Fishing effort (Figure 22) in the mid-Atlantic area is primarily from trips targeting squid species.  Compared to the New England groundfish fleet and the fleet as a whole, fishing effort in statistical areas 521, 537, and 539 were over represented in Study Fleet data, while statistical areas 513 and 514 were underrepresented based on the number of subtrips (Figure 23).

Many of the patterns observed in Study Fleet distributions reflect the very specific geographic locales of the three fleet sectors.  Both the Gulf of Maine and southern New England sectors correspond to regions with large fishing fleets, and in particular, large groundfish fleets.  The Cape Cod fleet sector covered a disproportionate number of vessels compared to the regional makeup of the NE groundfish fleet.  This can explain why the Study Fleet data were skewed towards the smaller, medium capacity vessels fishing longline gear to target haddock in statistical area 521.

The final Phase II fleet composition did not constitute a statistically representative subset of the NE groundfish fleet nor did it target only the high capacity vessels. However, in general terms, the geographic dispersion, vessel sizes, gears fished, and species caught covered the range of the NE groundfish fleet, though the strata distributions were not identical.  Based on 2004 groundfish landings, 20% of the fleet caught 72.9% of the total landings.  Under the first deployment strategy, this would require a fleet size of approximately 180 vessels (based on a total fleet of 895 vessels).  Under the second deployment strategy, the fleet size of a stratified fleet would be contingent on acceptable variance limits and constructing a fleet large enough to reduce the within-strata variance to an acceptable level.  These fleet analyses are intended to provide a general overview of the fleet composition and to compare the composition to two proposed deployment strategies.  It is necessary to consider deployment strategies as the Study Fleet moves from a pilot program to a production-program deployment.  The various deployment strategies need to be considered in conjunction with program objectives.

DISCUSSION AND CONCLUSIONS

The Study Fleet pilot program was successful in developing, testing, and deploying an ELB system among the New England groundfish fleet.  The system included hardware, software, data transmission, and in-house data storage components.  The haul-by-haul data collection resolution was finer than existing self-reported data collection programs (FVTR) and in some cases equal to that of the NEFOP.  This pilot program did show that more accurate self-reported haul-by-haul data can be collected by the fishing industry through the use of ELBs. Despite the program’s successes, there are areas requiring improvement and some notable limitations which should be considered when discussing continued use of the Study Fleet concept.

ELB System Recommendations

It is recommended that the future development of the ELB systems focus on the use of inexpensive laptops and further development of the UNH logbook.  The inexpensive PC laptops outperformed the more expensive ruggedized and tablet computers.  The UNH logbook has undergone more extensive field testing, has more user-friendly functions, and more importantly, can accommodate some fixed-gear fisheries. Data quality issues associated with UNH logbook V 4.0 should be addressed in future versions (Table 13, Table 14, Table 15, and Table 16). An attempt should be made to improve the logbook to allow for the recording of the setting and hauling of nontended fixed gear. This will require that fishing gear can be tracked separate of fishing trips (i.e., gear is set on one trip and hauled on a subsequent trips; fishing effort will begin on the first trip, but the fishing effort and resulting catch and landings must be attributed to the second trip).  It is recognized that there still may be fishery limitations on the use of logbook (i.e., entry of haul-by-haul information may still be problematic for gear hauled in quick succession on vessels with small crew sizes).

Because of extensive corrosion, the use of the ACR® Nautilus 85 temperature probes should be discontinued. Use of probes capable of transmitting data without requiring opening of the device should be investigated (i.e., optical, radio frequency data transmission). Additionally, a probe capable of recording depth may improve the accuracy and precision of collected data on fishing effort and should be considered (discussed in more detail below). Standard protocols on the deployment of probes on commercial fishing gear must be developed to ensure consistency of placement and data sampling rates across the fleet.

Based on the performance of the two tested VMSs, it is recommended that Boatracs® units be utilized to the extent practicable. It is recognized that because of cost and power limitations, a SkyMate® VMS may be the only option available to some vessels. A third VMS, the Thrane & Thrane SAILOR unit, was approved by the Regional Administrator for use in the Northeast Region on May 26, 2006 (http://www.nero.noaa.gov/nero/nr/nrdoc/nrphlo6/06ThraneVMS.pdf). Use of this VMS for transmission of logbook reports should be explored in addition to nonVMS methods of at-sea data transmissions (e.g., satellite modems).

A web-confirmation system is needed to allow fishers to view, edit, and confirm the data they have submitted. In-house data quality audits should be implemented to flag questionable data and bring these records to the attention of fishers when logging onto the web-confirmation system. A similar system has already been built by the NEFSC for use by a study fleet targeting northern shortfin squid (Illex illecebrosus) (Hendrickson et al. 2003).

Data Collection Summary and Recommendations

The haul-by-haul data collection of the Study Fleet pilot program offered several improvements over the existing FVTR program in the way of fishing effort characterization.  In particular, the capture of catch location was greatly improved, with Study Fleet statistical area reporting having a 90.6% agreement with NEFOP data (Table 5) and both suggesting more than 95% misreporting of trips fishing in multiple statistical areas in FVTR data (Table 6 and Table 7).  If misreporting of these multiple subtrip trips results in the assignment of species landings to an incorrect stock area, this could have significant implications on the underlying assumptions of many groundfish stock assessments. Haul-by-haul data collection programs such as the Study Fleet may offer improvements in these areas.

Haul-by-haul data allow for greater precision and the estimation of variability in haul duration. While the analyses of haul duration showed differences between the Study Fleet pilot program, FVTR program and NEFOP, it is difficult to discern the contribution of potential causes (differences in protocol, operator error, estimation error, etc.).  Noted limitations of the logbook with regards to capture of fixed-gear soak duration, and operator error in the use of the logbook contributed to inaccuracies in the Study Fleet data.  Fishers must be properly instructed as to the protocol for determining when hauls start and end to ensure consistency across the fleet.  Additionally, the number of efforts recorded in Study Fleet data was lower for most gear types compared to the NEFOP and FVTR data, likely because of a combination of logbook malfunction, user error, improper training, and limitation of logbook system on some fixed-gear vessels.  Methods should be investigated to validate the button activity of the logbook and provide QA/QC for collected number of efforts and haul durations.  Temperature-depth probes attached to the fishing gear could offer some improvement in these areas, as the depth signal could be used to validate the number of times gear was deployed as well as the duration of deployment.

The existing gear configuration data elements (gear, size, quantity, mesh size, mesh type) collected by the Study Fleet pilot program are equivalent to the FVTR requirements, with the single exception of mesh type.  Without expanding the gear elements collected by the Study Fleet, it will be difficult for the existing program to capture information necessary to assess the efficacy of new regulations (e.g., use of turtle excluder devices, raised footrope trawls) and Species Access Programs (e.g., haddock separator SAP).  To increase the potential uses of study fleets, the data elements collected by the ELB system should be expanded to include many of those elements collected by the NEFOP (URL: http://www.nefsc.noaa.gov/fsb/).  It is also recommended that standard protocols are developed for the measuring of these parameters and that fishers b sufficiently trained in the use of the protocols.

Kept catch reporting to the Study Fleet was relatively high (approx. 70%) when compared to NEFOP but was about equal with FVTR.  Study Fleet kept quantities were generally less than the other sources at both the haul and trip-levels.  Haul-by-haul reporting of discarded species appears to result in higher reporting rates as observed in the Study Fleet vs. FVTR comparisons. This result is consistent with conversations with fleet participants who mentioned the difficulty they have in recalling discarded species and weights at the end of a trip when filling out FVTRs. Working with fishers to implement catch estimation methodologies may help improve the captain’s hail weights of both the kept and discarded catch.  Subsequent training will be needed to focus on the importance of reporting all retained catch, such as lobster.  On-going training is needed to improve the reporting of individual species; use of generic species-complex codes such as red/white hake and skates should be discouraged to the extent practicable.  Standard protocols that will improve catch weight estimation yet are conducive to “normal” fishing operations must be developed.  Fleet participants should be used to develop these protocols cooperatively. Prior to deployment in a production-level Study Fleet pilot program, all crew members should be trained on the use of these protocols.

Study Fleet landings were generally less than FVTR landings; a finding consistent with lower hail-weights of the retained catch portion in the Study Fleet data. It is logical that low hail weights for individual hauls will results in lower trip sums and landings estimates. Self-reported (Study Fleet, FVTR) landings data were typically less than those of the dealer landings. There was a high percentage of nonreporting of landings and misreporting of nondealer transactions (home consumption, kept as bait, etc.) in Study Fleet data. Part of this misreporting may be due to difficulty in using the landings portion of the ELB software. Several fleet participants mentioned that they felt this portion of the software was difficult to use. Improvement of the logbook in regards to landings and ensuring that fishers understand that all landed catch must be reported regardless of whether it is sold to a permitted dealer could improve landings reporting in Study Fleet data.

In discussing the data collected through the Study Fleet pilot program it is important to consider the implications of compensating participants.  With the exception of the four months of voluntary participation, all participants were financially compensated.  Compensation provides an incentive for fishers to participate in training sessions, continue to use and troubleshoot the developing technology, and make themselves and their vessels available to field technicians for regular field visits. Additionally, compensation ensures that collected data meets certain criteria (i.e., if performance is not adequate, compensation can be discontinued).  It is uncertain if data quality would continue at the present level without on-going compensation to participants. A certain level of data quality can be assured by building QA/QC controls into the ELB software; however, these can not address all data quality issues.  Given the burden of haul-by-haul reporting on the fishers, it is likely that some level of compensation would be necessary to ensure adherence to protocol and the quality of collected data.

Recommendations on Future Fleet Deployments

The ELB system is operational and recommended for future deployments in study fleets. The ELB system has been tested on the spectrum of vessel types participating in the New England groundfish fishery in addition to several other fisheries (e.g., squid, lobster pot, scallop dredge). While the composition of the Study Fleet pilot program was not identical to the NE groundfish fleet, the ELB system was deployed on a variety of vessels sufficient to develop an understanding of the strengths and weaknesses of the technology. The system is best suited for deployment on vessels fishing mobile gear or tended fixed gear (e.g., some longline, gillnet, and pot fisheries).  Future deployments of the system on study fleets should be done with a clear understanding of the limitations of the technology, but more importantly, consistent with future Study Fleet program objectives. While only two objectives are discussed here (quantifications of fishery removals and characterization of a larger fleet), there are other deployment strategies that should be considered including census coverage on small fisheries (e.g., tilefish, red crab) and using study fleets to cooperatively address specific research questions.

ACKNOWLEDGMENTS

We would like to thank all Study Fleet participants, the Gulf of Maine Research Institute, the Cape Cod Commercial Hook Fishermen’s Association, and the Manomet Center for Conservation Sciences.  Additionally we would like to acknowledge the considerable amount of work performed by Perot Systems staff and subcontractors in developing and field testing these systems, in particular, Carl Leoncello, Story Reed, and Ted Racine, David Miller, Ayhan Ergul, Scott Tibbetts, and Greg Hanson.  There were many staff members from the NMFS-Northeast Regional Office who were instrumental in the planning, developing, and managing of this project, including Earl Meredith, Nick Anderson, Sarah Pike, and Ken Ortiz.  Lastly we would like to thank Steve Murawski; without his vision and dedication to the Study Fleet concept, this project would never have seen the light of day.

Endnotes

1.  Technology Planning and Management Corporation was later purchased by Perot Systems.

2.  Fishing Vessel Trip Report instructions are available on-line at http://www.nero.noaa.gov/ro/fso/vtr_inst.pdf

3.  Thistle Marine^TM, Ellsworth, Maine.

4.  P-Sea WindPlot© II, P-Sea Software© Co., Morro Bay, CA.

5.  Boatracs® FMCT/G Installation Guide.  Copyright 1990 - 2003.

6.  SAS Institute Inc., Cary, NC.

7.  The 4.8% matching rate is consistent with the 5% mandated NEFOP coverage for New England groundfish fisheries in 2005.

8.  The ELB has the flexibility to capture both individual and combined haul information, but the SFLEET database does not.

9.  Area fished was not a required data element in the dealer data after May 1, 2004.

REFERENCES CITED

Bucklin A, Goudey C, Wiebe PH. 2001. FleetLink: collection & telemetry of ocean and weather data. Sea Techn. 42(5):27-31.

Gallaway BJ, Cole JG, Martin LR, Nance JM, Longnecker M. 2003a. Description of a simple electronic logbook designed to measure effort in the Gulf of Mexico shrimp fishery. N. Am. J. Fish. Mgmt. 23:581-589.

Gallaway BJ, Cole JG, Martin LR, Nance JM, Longnecker M. 2003b. An evaluation of an electronic logbook as a more accurate method of estimating spatial patterns of trawling effort and bycatch in the Gulf of Mexico shrimp fishery. N. Am. J. Fish. Mgmt. 23:787-809.

Gulf of Maine Aquarium. 2001. Study fleet and industry based survey meeting summary November/December, 2000. On-line document: http://www.nero.noaa.gov/ro/doc/sfibsfr.pdf.

Hendrickson LC, Hiltz D, McBride HM, North BM, Palmer JE. 2003. Implementation of electronic logbook reporting in a squid bottom trawl study fleet during 2002. Northeast Fisheries Science Center Reference Document 03-07, 30 p.

Murawski SA. 2002. Statement to the US Commission on Ocean Policy: Scientific challenges in supporting living marine resource management. On-line document: http://www.nefsc.noaa.gov/misc-docs/Oceans_commission_murawski.pdf.

Murray KT. 2005. Total bycatch estimates of loggerhead (Caretta caretta) turtles in the 2004 sea scallop (Placopecten magellanicus) dredge fishery. Northeast Fisheries Science Center Reference Document 05-12, 22 p.

[NEFSC] Northeast Fisheries Science Center. 2005. 41st Northeast Regional Stock Assessment Workshop (41st SAW): 41st SAW Assessment Report. NEFSC Reference Document No. 05-14. 237 p.

Wigley SE, Terceiro M, DeLong A, Sosebee K. 1998. Proration of 1994-96 USA commercial landings of Atlantic cod, haddock, and yellowtail flounder to unit stock areas. Northeast Fisheries Science Center Reference Document 98-02, 32 p.

Zar JH. 1999. Biostatistical analysis. 4^th ed. Upper Saddle River (NJ): Prentice Hall.

ACRONYMS

ASCII = American Standard Code for Information Interchange
CFR = Congressional federal register
CFDBS = Commercial Fisheries Database System
CPUE = Catch per unit effort
ELB = Electronic logbook
EVTR = Electronic fishing vessel trip report
FMP = Fisheries management plan
FVTR = Fishing vessel trip report
FIPS = Federal Information Processing Standards
GPS = Global positioning system
ITIS = Integrated Taxonomic Information System
LPUE = Landings per unit effort
NAFO = Northwest Atlantic Fisheries Organization
NE = New England
NEFOP = Northeast Fisheries Observer Program
NEFSC = Northeast Fisheries Science Center
NMEA = National Marine Electronics Association
NERO = Northeast Regional Office
NGO = Nongovernment organization
NMFS = National Marine Fisheries Service
NOAA = National Oceanic and Atmospheric Administration
OBDBS = Observer Database System
PC = Personal computer
PSW = P-Sea WindPlot©
QA/QC = Quality assurance/quality control
SARC = Stock Assessment Review Committee
SFLEET = Study Fleet Database System
UNH = University of New Hampshire
VMS = Vessel monitoring system