Do ARM Measurements "Make-a-Difference" When Used in Carbon Models?
One simple way to evaluate and demonstrate the utility of ARM measurements for use in carbon models is to compare directly the outputs of carbon simulations using ARM data with outputs of the same simulations not driven with ARM measurements, quantifying any differences which result. In developing non-ARM alternatives for comparison and relative evaluation, we used the following rhetorical question as a guide: "What would an intelligent carbon modeler do in the absence of ARM data for simulating a location inside the ARM CART region?"
One plausible answer to this question is to use any of several synthetic weather generator utilities to drive the simulation. These simulations are said to produce statistically realistic time sequences of meteorological parameters, given a temporal string of actual weather measurements or a set of statistical climate normals for a particular location as input. Synthetic weather generators are a commonly used alternative to meteorological measurements; in fact, one popular carbon model (Biome-BGC) uses the same driver data input format as the output from one such synthetic climate generator (Mt-Clim) in order to facilitate their sequential use.
The figure above outlines the MAD framework, which is a specialized type of model sensitivity analysis designed specifically to test the effect of including ARM archive measurements. In its simplest form, this "Make-A-Difference" (MAD) framework conceptually consists of two parallel pathways to be compared. The right vertical track begins with actual ARM measurements, while the left track begins with synthetically generated meteorological time series. The two alternative pathways are compared both before and after the model, as differences in both the input data sets and the output data sets that result. In particular, synthetic weather generators may not duplicate clouds correctly, thus creating differences in radiation which may be important to photosynthetic carbon fixation. The red horizontal arrows indicate places where the two alternative pathways are examined for differences.
Comparison both before and after the model allows us to determine how sensitive a particular simulation is to differences in its input stream, if present. A model may be relatively insensitive, in terms of particular outputs, to rather large changes in some input values. For such values, having measurements rather than estimates may add little value. However, even small changes or estimation errors in some inputs may produce large differences in outputs. The use of actual measurements for this sort of driver input may result in substantial improvement in the quality of simulated carbon outputs.
Since detailed measurements of carbon pools and fluxes are not available at the ARM CART, we cannot know which of the pathways is more accurate, and so we must be satisfied with the demonstration of differences. Ostensibly, the use of measurements rather than synthetic values should produce more accurate carbon flux and pool estimation results.
We have used the MAD framework to evaluate the value added by using ARM measurements to drive the Biome-BGC model (click here to review a description of Biome-BGC).
In this implementation of the general MAD framework, we duplicated the left (no-ARM) pathway to consist of three alternative methods that a carbon modeler might use to drive the Biome-BGC model:
Although many carbon simulations are now driven with hourly data, Biome-BGC is driven by daily changes in meteorology. That only daily weather data are needed makes it possible to drive this model using two different popular synthetic weather generators: MT-CLIM and GEM-Shaw. MT-CLIM (Mountain Climate Simulator) was originally developed by Running et al. (1987), and version 4.3 has been substantially modified by Dr. Peter Thornton, currently at NCAR. MT-CLIM deterministically "adjusts" temperature, vapor pressure, and solar radiation values supplied at a "base" location to values appropriate for a different "site" location, based on latitude, elevation, and mean yearly precipitiation. GEM (Generation fo weather Elements for Multiple applications) was developed from the basic internal structure of USCLIMATE and WGEN, but includes significant improvements. GEM uses the 30-year climate normals for a number of fixed locations as input to stochastically generate synthetic weather streams, and is the weather generation tool recommended by the USDA NRCS and ARS.
Starting with the leftmost pathway, we used the 30 closest Oklahoma MESONET weather stations to provide spatial replicates of input for the deterministic MT-CLIM weather generator, which then produced 30 different replicates of the climate at the ARM Central Facility. In the middle pathway, we used the GEM model, driven by climate normals from the Oklahoma City airport, to produce 30 different replicates of the climate at the Central Facility. This pathway assumes that the Ok. City airport is "close enough" to adequately represent the climate at the Central Facility. Finally, we used a sequential combination of both synthetic generators; GEM to generate 30 replicates of Ok. City airport climate, each followed by an application of MT-CLIM to "adjust" the climate to the Central Facility location. In this way, each of the three alternative pathways was used to produce 30 independent replicates of possible synthetic climate with which to drive the Biome-BGC simulation.
Use of ARM measurements to drive Biome-BGC did make a difference relative to driving the model with synthetic meteorology, as is commonly done. On the input side, daylight average shortwave radiation was consistently and significantly underestimated by all of the synthetic climate generators tested. Both the yearly median and the cumulative measurements were greater than the 5th-to-95th percentile ranges for most simulated days. Underestimates in shortwave radiation probably contributed to downstream underestimates of photosynthetic production as compared with ARM measurements.
Use of ARM data also produced dramatic effects on Biome-BGC outputs. Climate forcings from all of the synthetic generator pathways led to underestimates of leaf-level carbon assimilation. Net Ecosystem Production (NEP) was highly variable from year-to-year. In some years, the median NEP is always positive, but in other years it dips far below the zero line. Although such variations were shown in the populations from the synthetic climate estimators, the trajectory of NEP in any single year was strongly dependent on the exact sequence of hot, dry days occurring in late summer. Differences in the specific late-summer climate trajectory accumulated and were often significant throughout the five-year simulation.
Full results are available at http://research.esd.ornl.gov/~hnw/ARMCarbon/, and were also presented at the Proceedings of the Twelfth ARM Science Team Meeting, where they are available as a pdf document. A color poster highlighting these results was presented at the ARM Science Team Meeting and the MODIS Modland working group.
We have future plans to repeat the MAD framework for SiBD. SiBD is a more elaborate model than Biome-BGC (click here for a review of SiBD), and, unlike Biome-BGC, SiBD is driven with hourly or half-hourly meteorological information. SiBD makes use of more detailed solar radiation information (all of which is available as ARM measurements), so that the potential exists for large differences to exist when actual measurements of these radiation parameters are used in the model. This repetition of the MAD framework is well-underway, and some early SiBD output results are already available. As can be seen, the number of available outputs from this model is impressive.
The MAD framework provides a blueprint to evaluate differences that is not limited just to carbon models. Now that it has been developed, the MAD framework can be used to demonstrate the value of using ARM measurements in other disciplinary arenas as well (i.e., water cycle modeling). We will employ the same basic MAD framework to evaluate the impact of various gap-filling techniques in another phase of this project. We believe that the MAD framework represents a powerful, transplantable litmus test for assessing model sensitivity to improvements in input data.
