Appendix 2
ANALYSIS OF KNOWLEDGE GAPS
REGARDING FIRE AND NONNATIVE INVASIVE PLANTS
(Part of Final Report
INVASIVE PLANT AND FIRE INTERACTIONS:
USE OF THE FIRE EFFECTS INFORMATION SYSTEM
TO PROVIDE INFORMATION FOR MANAGERS
JFSP task 00-1-2-09 (PI: Kevin Ryan)
JFSP task 03-4-2-03 (PI: Jane Kapler Smith))
The following tables describe the quantity and quality of information available on topics essential for understanding the relationship between an invasive species and fire. The tables cover only the 60 species covered by 43 literature reviews for this JFSP task. This information is being prepared for submission to a peer-reviewed journal.
Methods: A single outline (table 3) is used to ensure completeness and consistency of information in all FEIS plant species reviews, including those for nonnative invasive species. While planning and writing the 43 species reviews for this task, we used the FEIS outline to keep track of knowledge gaps. First, we identified sections and topics crucial for understanding the plant’s relationship to fire (shown in bold print, table 3). Then we added questions related to prescribed burning to this list: Does any research describe effects of experimental fires in the field? Does any research describe effects of fires of different severities, fires in different seasons, or fire treatments repeated at different intervals? Does research describe fire effects after the 1st postfire year? As we wrote each species summary, we used key phrases, such as “research is needed” and “incompletely understood,” to identify knowledge gaps. A subsequent search of completed summaries for these key phrases pinpointed topics with knowledge gaps.
Knowledge gaps were often attributable to lack of information, but some occurred because the available information covered only a narrow range of conditions or a small geographic area. Knowledge gaps also occurred when information was of uncertain quality, such as anecdotal evidence and assertions unsubstantiated by data. To describe the source of knowledge gaps more systematically, we developed a scale representing the continuum of information quality:
4 Evidence from primary research published in a peer-reviewed journal
3 Evidence from primary research published in a technical paper from a research group in a state or federal agency, thesis or dissertation, book chapter, proceedings, or flora
2 Other substantial, published or unpublished experimental or observational data
1 Assertion with no experimental or observational data (that is, the source of the assertion is unknown)
0 No information or assertions at all
The highest value in the information quality scale (4) represents primary research published in peer-reviewed journals; for these articles, the population, variables, and scope of inference are usually well described, and blind peer review indicates the knowledge is probably reliable. An information quality value of 3 represents similar information published in an outlet that is reviewed by peers, but not anonymously. We classified publications ranked 3 and 4 as “high-quality” information. A value of 2 represents reports that have not been reviewed, as well as information reported without a description of rigorous scientific procedure (that is, not containing hypotheses, controls, replication, or statistical analyses), and thus having unknown certainty and scope of inference. A value of 1 represents knowledge presented without data, scientific procedure, or review and therefore potentially poor in quality or with an unknown scope of inference. Such information would include anecdotal knowledge and casual observations, for which the scope of inference is poorly defined or not described at all.
The information quality scale is simple and subjective, so it oversimplifies a very complex concept. We recognize many shortcomings in this scale. For example, blind peer review does not guarantee accuracy even though we ranked the quality of peer-reviewed articles as high (4). Additionally, a single peer-reviewed study may not provide sufficient information to support confident, widespread application even though it is ranked as “high-quality” (3 or 4) on the information quality scale. On the other hand, a manager may possess bountiful, accurate, unpublished data (ranked 2) or accurate anecdotal information (ranked 1) that applies directly to management. If a manager has been using fire to control a species for many years and reports that it does not sprout after fire, the information may be highly reliable even though we give it a rank of 1 or 2. Likewise, a senior researcher might make an assertion based on decades of experience without providing data to back it up; this citation would receive a value of 1 or 2 on our scale (depending on the context of the assertion) even though the information is probably reliable. When it was clear that the assertion was based on "substantial, published or unpublished experimental or observational data" we gave it a 2. If the source of the assertion was not given or otherwise unclear, we gave it a 1. Our scale, while somewhat subjective, is designed to show general patterns in information quality for the particular species discussed here.
We examined the knowledge gaps in each species review using the information quality scale. For botanical and ecological information, we recorded the highest quality of information available on each topic. For fire-related information, we identified the highest quality of information available on each topic and also tallied the total number of citations, of any quality, available on each topic.
Results: Information on three crucial topics for understanding relationships between fire and nonnative invasives (table 3) was adequate for nearly all species summaries: life form, seed production, and aboveground phenology. The few knowledge gaps identified for these topics include data on seed production of Oriental bittersweet and Russian-olive, and information on phenology of giant reed, privet species, and climbing ferns.
For the remaining topics identified as “crucial” in table 3, knowledge gaps in the literature were identified. Table 4 shows the highest quality of information found for biological/ecological topics in each species review, based on the information quality scale. For example, seed dispersal for Acer platanoides, Norway maple, is described by at least 1 article ranked 4, that is, containing primary research and published in a peer-reviewed journal.
Table 5 describes the quality and quantity of information found on fire-related topics for each species review. Information quality is expressed as in table 4, that is, the highest rank given any citation for that topic and review. Quantity is expressed (in parentheses) as the total number of sources cited in the species review. For example, 2 sources provided information on postfire seedling establishment of Cirsium vulgare, bull thistle; the highest information quality rank of these citations was 3, indicating at least 1 source described primary research and was published in a technical paper, thesis, dissertation, book chapter, proceedings, or flora.
Table 6 describes the quality and quantity of information on experiments reporting fire effects and using prescribed burns. High-quality studies are available in the literature for many species, but they seldom report effects of variation in important aspects of the fire regime (fire severity, seasonality, etc.). For example, 4 sources described fire experiments for Genista monspessulana, French broom, but none of these included information on varied fire severities or seasons, and only 1 experiment included information on varied burn intervals or reported results beyond the first postfire year.
Summary: High quality information was available for most species on topics related to basic biology and ecology, and especially on topics relating to distribution and site characteristics, seed dispersal, and post-injury regeneration (see pie charts summarizing data at end of Table 4). More than 10% of species reviews lacked information on seed banking, successional status, and seed bed requirements, and almost half of reviews lacked information on belowground phenology. Overall, most species reviews appear to have sufficient high-quality information on biological and ecological traits to make a preliminary assessment of the potential for establishment, persistence, and spread of those species in a particular area after fire. However, managers would be better equipped to make such an assessment with information on how those traits are expressed specifically in response to fire. Additionally, this information does little to inform managers on the potential for these species to alter fuel characteristics and fire regimes in an invaded area.
Information is much less complete for topics relating to fire than for topics related to basic biology and ecology (see pie charts summarizing data at end of Table 5). Less than half (0% to 44%) of species reviews had high-quality information on topics related to postfire response, and less than a fourth had high-quality information on fuel characteristics or fire regimes. Of the species reviews containing any information on these topics (high or low quality), several referred to only one or two sources (see bar charts summarizing data at end of Table 5). For example, while 17 species reviews contained some information on heat tolerance of seed, 9 of these reviews obtained this information from only 1 source, and only 6 reviews referred to more than 2 sources for this information.
While more than half of species reviews included information from fire experiments (see pie charts summarizing data at end of Table 6), writers found few species for which information from multiple experimental studies was available (see bar charts below pie charts). Because studies from different locations or seasons often have conflicting results, the lack of multiple studies covering fire effects could lead managers to apply results from 1 to 2 studies with unwarranted confidence. Few of the fire experiments reported in the literature tested species responses to varied fire severities or burn intervals, and only about one-third covered variation in season of burn or extended beyond the first postfire year.
The information in this Appendix is being integrated with more detailed analyses of these knowledge gaps and their implications to fire and invasive species management, including recommendations for preparing and using literature reviews. It will be submitted to a peer-reviewed journal.