The ACPI Avant Garde Project
Principal Investigator:  Ian Foster (Argonne National Laboratory / University of Chicago)
DOE Co-Investigators:  Chris Ding (LBL), John Drake (ORNL), Phil Jones (LANL), Jay Larson (ANL), Doug Rotman (LLNL)
NCAR Co-Investigators: Maruice Blackmon (CGD), Byron Boville (CGG), Tom Bettge (CGD) Cecelia Deluca (SCD), Dave Williamson (CGD)

Project Description

This project is one of two experimental projects for the Accelerated Climate Prediction Initiative (ACPI).  Our focus is on global climate modeling, while the other project (led by Tim Barnett of SCRIPPS) is focused on regional climate modeling.  Our goal is the creation of a performance-portable parallel coupled climate system model.

The Climate System Model (CSM-1) and Parallel Climate Model (PCM-1) of the National Center for Atmospheric Research (NCAR) are two advanced climate models that have seen significant use.  CSM-1 is the first community-based model to link atmospheric, oceanic, biologic, cryogenic, and chemical components; it has been and continues to be used for a wide range of climate research.  Developed with DOE support, PCM-1 couples similar models and, in addition, has been adapted to execute on scalable parallel computers, hence allowing long-duration simulations in support of DOE missions.

Recognizing the strengths of these two models, NCAR scientists are merging the CSM-1 and PCM-1 code bases to produce CSM-2, with the goal of achieving significant improvements in simulation quality.  However, despite its scientific strengths, CSM also suffers from two serious software engineering limitations.  First, the model has not been designed to exploit the microprocessor-based scalable parallel-architecture computers that are currently being deployed at NSF and DOE centers.  A consequence of this limitation is that CSM performance has not increased substantially in the past five years.  Second, the model’s structure has not been designed with a view to enabling “plug and play” substitution of important modules, such as dynamical solvers, physics packages, and model components.  This limitation both complicates ongoing CSM development and makes it harder for intramural and extramural scientists to experiment with improvements to CSM components.

Recognizing both the scientific strengths of CSM and the importance of overcoming these limitations, a group of DOE and NCAR scientists are pursuing a joint R&D activity aimed at developing a next-generation modular, performance-portable CSM-2.  This work is expected to produce two primary outcomes: a series of performance-enhanced versions of CSM-2, better able to exploit microprocessor-based parallel computers, and a detailed design for current and future CSM versions that improves substantially over current practice in terms of modularity and portability.  A substantial challenge in both these areas will be to improve CSM software engineering practices without unduly disrupting CSM-2 development or diverging from a common code base.

This  research and development activity will tackle the design and development of (1) a next-generation coupler and (2) a scalable, modular atmosphere model.  We choose these tasks because these two CSM components are the most complex and the most in need of enhancement.  Work on the coupler will address issues of scalability and configurability.  Work on scalability is important because the current coupler is a significant performance bottleneck in today’s CSM, preventing the model scaling beyond around 64 processors.  Work on configurability is important because CSM users want to be able to use CSM components in a wide variety of modes, which the current coupler design makes difficult.  In the atmosphere domain, work will focus on improving node performance, developing a more modular atmosphere model structure that permits the substitution of both dynamics and physics components, and developing the high-performance communication libraries required for good performance on scalable parallel computers.  In addition, the DOE/NCAR team will also work on improving ocean model and I/O performance on parallel computers.

This project has concrete, realizable tasks and milestones for both DOE and NCAR participants.  These activities support the scientific directions for CSM, as defined by the CSM scientific steering committee, and also the goals of DOE’s CCPP Program.  Day-to-day activities are coordinated by a small management group,  working closely with the recently formed CSM Software Engineering working group.  It is expected that participants in this project will become active participants in that working group.

This project is an 18-month activity.  We believe that this is an appropriate timeframe in which to be tackling these challenging CSM design and implementation tasks: a shorter project could not make useful progress, while a more ambitious but longer project might not have the required immediate impact on CSM development.  However, we emphasize that the task of enhancing CSM for modular development and scalable parallel execution is a substantial project and while much can be done within 18 months, the effort will certainly not be completed at that time.