EOS Data Information System (EOSDIS) Panel Meeting
- David M. Glover (dglover@whoi.edu), Chair EOSDIS Panel, Woods Hole Oceanographic Institution, Woods Hole, MA
A meeting of the EOSDIS Panel was called before the May 1996 IWG to discuss "hot" issues surrounding EOSDIS development. The meeting was held at the Hughes facility in Landover, MD. Approximately half the panel attended the meeting plus members of ESDIS and NASA Headquarters.
This report will cover the election of the chair, a report from the Ad Hoc Working Group on Consumers (AHWGC), the latest federation plans (including a brief discussion of what transpired at the Working Prototype [WP]-Earth Science Information Partner [ESIP] workshop June 1996), a discussion of the future directions of the EOSDIS Panel, a brief update on the Community Cost Model (CCM) being worked on by Bruce Barkstrom, a briefing by Skip Reber on the formation of the EOSDIS Resource Management Board, and a report on the Independent Cost Evaluation (ICE). It was hoped that some of this discussion would help frame the issues and questions regarding EOSDIS at the Payload Panel meeting (29-31 July 1996).
Many requests for nominations were made via e-mail, but all replies indicated that the incumbent chair should stand another watch. Therefore, David Glover has, by acclamation, been re-elected chair of this Panel for two more years.
Dave Emmitt provided a summary of the AHWGC report that H. K. Ramapriyan (April 25, 1996) circulated via e-mail. Emmitt's summary was based on Matt Schwaller's analysis of the 2X (i.e., 2 times) resource limit presented to the ESDIS project in January 1996 and the AHWGC report. The main conclusion is that as Instrument and Interdisciplinary Science (IDS) Teams claims/demands go down, it looks as though 2X will be enough to satisfy the needs of the EOS community and is a reasonable initial design target. If you go outside of the EOS community, however, all bets are off since estimates vary from as high as 33X to the nominal 2X in addition to EOS community demands. This is important because in the "even reasonable to think about" range (2X to 64X) it comes in at $20-40M/X.
A lot of statistical analysis was applied to the data the AHWGC had collected about the "pull" side of the user model. But given that 90% of the pull is on 10% of the data (as expected), it was suggested that EOSDIS find some way to regulate the pull on the system. It was a conclusion of the AHWGC report that EOSDIS should begin planning now to manage demand that exceeds capacity. They further recommend access be handled through a "charge for timeliness" of delivery.
Among other "access" issues was the issue of the "standard" data format. This issue is being re-visited by ESDIS and a new policy was circulated by Ramapriyan in a July 8, 1996, e-mail message.
Of the two recommendations from the National Research Council (NRC) to NASA about EOSDIS, the recommendation that EOSDIS be reconfigured "to transfer responsibility for production generation, publication, and user services to a competitively selected federation of partners in government, academia, and the private sector" has raised many concerns. Much of the activity NASA undertook to respond to this recommendation (and the NRC) is documented on a world-wide web (WWW) page (http://www.hq.nasa.gov/office/mtpe/eosdis/). There was concern that NASA was moving too rapidly towards a solution/response and the instruction from the NRC this spring to slow down was met with relief. The Response Task Force (RTF) disbanded at the end of May 1996, its functions having been identified and farmed out.
At the EOSDIS Panel meeting and again later at the WP-ESIP workshop the topic of just what is being federated came up. As we understand it now, the federation of EOSDIS will be carried out in two phases, with none of the measurements from the suite of 24 core measurement types "federated" until after 1999. Earth Science Information Partners (ESIPs) will come in three varieties: Type-1 ESIPs will be those providing data whose product has high reliability, maintainability, availability (RMA); Type-2 ESIPs will provide data that's not available from Type-1 ESIPs due to its "not-ready-for-primetime" nature; and Type-3 ESIPs will be those providing data beyond the global change research community, potentially for profit. In Phase I, working prototypes of Earth Science Information Partners will be competitively selected, but only for Types-2 and -3. In 1999 there will be a competition for all three types of ESIPs. In the interim, DAACs will be recertified by a panel of outside reviewers with three possible outcomes: recertification, one-year probation, or phase out. A schedule for recertification, Cooperative Agreement Notice (CAN) for WP-ESIPs, and all-out competition for all data products is available at the above WWW site.
The purpose for these WP-ESIPs is to explore the possibilities of federation. How should (can) a Federation be formed? What issues should be on the table for their consideration? How will they adjudicate disputes within and with outside competition for resources? All of these, and many more, are important questions that need answers based on experience. Hence, it seems reasonable to explore the idea of working prototypes. In principle, these WP-ESIPs are testbeds so that lessons learned from these experiences can be transmitted to the actual, operational ESIPs of the next decade. But these WP-ESIPs must be working prototypes, they must support at least three things: real science, user services, and technological innovation. Real science is required because these WP-ESIPs will provide a real service to the global change research community by providing data products that are too developmental to be supported at a DAAC/Type-1 ESIP. At the time of this writing it is not known whether or not funds for the Type-2 ESIPs will be available to support this science. User services are required to explore ways in which the productivity of global change research, and other Earth Science data use activities, can be improved. Technological innovation is required to help NASA explore ways in which new technologies in data management and distribution can be used to bring down the cost of EOSDIS. All of these activities will be supported and encouraged through the formation of a federation. But, most importantly, it is imperative that these WP-ESIPs be allowed to fail. For without failure these ESIPs will be more of the same, and no new knowledge on how to form a database economy will be gained.
The Panel felt that the direction NASA is now pursuing, with this new federation of data providers, is actually closer to what we originally had in mind back during Phase B. Independent, competitive but cooperating data providers are the essence of what is required for an evolving, extensible, physically and logically distributed data system.
At first the goal of the EOSDIS Panel was to ensure the establishment of a good architecture (extensible and distributed). With the arrival of the EOSDIS Core System (ECS) we feel that goal has been, to a large extent, accomplished. The ECS software delivered by Hughes is not dependent upon the somewhat logically centralized structure imposed upon the DAACs by funding realities of NASA. This should mean that, come what may (federation or whatever), the architecture of EOSDIS will survive into the next decade. How well it will be able to adapt (evolve) to changing conditions is something that only natural selection and time will be able to evaluate.
The goal the EOSDIS Panel sees before it now is the creation of a federation of data providers, much talked about but with little experience to go on. While we realize that we cannot singlehandedly create a federation and that a lot of work will necessarily be done by NASA and these WP-ESIPs, we wish to provide whatever insight and help that we can. During our meeting in Landover we tried to identify the key issues that we felt surrounded a successful establishment of a federation.
Primary among the issues discussed was the problem of how to bootstrap a federation into existence. What will be the "glue" that binds the separate and independent data providers together? The dictionary definition of a federation is "a group united by a common agreement under a central government or authority." Ignoring, for the moment, the fact that NASA will remain the central authority as long as it provides the money, what could/will this common agreement be? Absent from the draft CAN (reviewed at the WP-ESIP Workshop) was any clear mechanism to engender such an agreement. In short, the ESIPs have got to want to federate for reasons other than that they were told to.
Other issues were also discussed. What should the governance structure of this federation be? How shall they adjudicate resource allocation contention? What about innovative ideas that were not tried, or were cut due to budget pressures? Can this be a way to bring them back long enough to test them? Where do the ESIPs interface with ECS? Is it up to them? What sort of success criteria should be used and how does a negative result, i.e., failure translate into positive information (we'll know better not to try that next time). Plenty of questions and not a lot of answers, but the answers are what the WP-ESIPs are to find. The CAN needs to clarify the questions NASA is asking.
Bruce Barkstrom briefed the panel on the status of the Community Cost Model (CCM). During this briefing a discussion about an interface between the CCM and WP-ESIPs came up. It was suggested that proposers to the WP-ESIP CAN provide an input file to Barkstrom's CCM so that their system's cost could be compared against the current EOSDIS cost. This would provide a metric against which NASA could judge whether or not a WP-ESIP was really reducing costs. The CCM is currently up to chapter 19 and is available on the web at http://asd-www.larc.nasa.gov/cost_model/doc.html.
Skip Reber briefed the Panel on the formation of an EOSDIS Resource Management Board (ERMB) by NASA. In many respects the ERMB raises issues similar to the ones being investigated by the Ad Hoc Working Group on Production (AHWGP), but with an emphasis on coordinating and translating this information into real costs. The goal of the ERMB is to keep track of resource requirements for the generation of EOS data products, manage and allocate budget caps if necessary, and encourage Instrument Teams (ITs) to be more efficient with their processing scenarios to stay below any spending cap that may appear in the future. The ERMB was just in the early stages of being formed and discussions were held concerning of acceptable units for resource management (dollars, clock time, etc.), how to acquire an accurate assessment of resources required, what should the mechanism for resource allocation be, and longer term monitoring and management strategies.
In order to fit a 1.5-hour briefing into a half hour we were briefed by Rich Saad on only one DAAC (GSFC). The costs for the GSFC DAAC are right on target when compared to databases of very similar organizations. The results of the three subcontracts were reviewed in their assessment of flight operations, data capture, and science data processing and archiving. "Should costs" for these components were based on common industry practice with a yearly inflation rate of 3.5%. The overall results for GSFC showed that the Independent Cost Evaluation (ICE) was different from the NASA cost estimates by only -3%, the cost model they used had an error range of ± 10%.
There was some discussion that the "should cost" approach used by ICE was based on current industry practices and does not reflect any innovative strategies for reducing cost. Nevertheless, in a "business as usual" sense, the ICE report seems to indicate that EOSDIS costs no more than any other industry effort of similar size, scope, and nature.
Additional presentations were made by H.K. Ramapriyan on the current federation plans, Joy Colucci on Hughes Science Office status, Paul Fingerman on ECS software reuse strategies, and Menas Kafatos on interfaces and tools for interdisciplinary science. Much of what Ramapriyan presented provided stimulation for what was discussed above. Colucci's presentation was basically an update on science software integration and test, AHWGP studies, data migration to HDF, and incremental track progress. Fingerman made an abbreviated presentation (due to the lateness of the day) of how parts of ECS can be reused at other locations, demonstrating its flexibility. Kafatos presented "Interfaces and Tools for Interdisciplinary Science in EOSDIS," which was a description of the Virtual Domain Application Data Center (VDADC) engine. The VDADC is an outcome of the George Mason University (GMU) architectural study of 1994 and provides for specialized centers where user communities can access EOS and other Earth science data without overloading EOSDIS.
[Table of Contents]
[Previous]
[Next]