| |
The
Egyptian god Djehuty was the guide to heaven, earth, and the netherworld;
lord of calculation, wisdom, and judgment; and protector of knowledge,
mathematics,
and science. His image is seen in many hieroglyphic tablets.
IN
the Egyptian pantheon, Djehuty was the guide to heaven, earth, and
the netherworld; lord of calculation, wisdom, and judgment; and
protector of knowledge, mathematics, and science. It seemed appropriate,
then, for Lawrence Livermore astrophysicists David Dearborn and
Peter Eggleton to take his name for their breakthrough three-dimensional
code that simulates the evolution and structure of stars.
 The physical processes of
stars have long been of interest to Livermore researchers because
understanding the prime mechanism of stellar energy— thermonuclear
fusion—is part of the Laboratory’s national security mission.
“Stars are high-energy-density ovens,” says Dearborn.
“Several Laboratory programs are interested in the properties
of stars, and many Livermore physicists have backgrounds in astrophysics.”
Dearborn points out that
stars provide the standards of reference for measuring the size,
age, chemical composition, and evolution of the universe. Stars
have also been used as physics laboratories that strengthen our
understanding of complex physical processes. For example, they have
been used to better understand the properties of hot plasmas as
well as fundamental particles such as neutrinos. Stars have also
been used to suggest the properties of exotic particles such as
axions, which have been proposed to explain why the universe contains
more matter than antimatter.
Eggleton
notes that scientific knowledge of stars may appear to be mature,
but in fact, much of what we know about stars—especially the
way they generate energy and how they evolve from a dust cloud to
a supernova or red giant—may well be significantly incomplete.
“We need to improve our knowledge about stars,” he says.
The
reason for the imperfect understanding is that many stellar processes
are complex, three-dimensional phenomena that have been modeled
only in coarse approximation using one-dimensional computer codes.
For example, the transport of energy through a star by convection
from its superhot core is a three-dimensional process, which limits
the value of one-dimensional calculations, even for perfectly spherical
stars. (See the box below.) Although a one-dimensional convection
simulation could be inaccurate by only 10 percent at any moment
in time, such “small” errors can easily accumulate over
time. The result might be a final discrepancy of 100 to 200 percent
in some properties calculated for such stellar objects as Cepheids,
which are large, pulsating stars often used to calculate the distance
scale of the local universe.
|
Probing
the Interiors of Stars
|
|
Stars, unlike planets, produce their own energy and
do so by thermonuclear fusion. Much of the complexity
underlying the computer code Djehuty, Livermore’s
three-dimensional code for star structure and evolution,
is its realistic simulation of fusion, which converts
hydrogen nuclei into helium ions. The process is often
called hydrogen burning and is responsible for a star’s
luminosity.
Fusion reactions
occur in the core, the innermost part of the star. In
a star about the size of our Sun, the hydrogen fuel
is eventually consumed after billions of years. The
core slowly starts to collapse to become a white dwarf
while the envelope expands to become a red giant. Our
Sun will reach this stage in about 5 billion years.
In contrast,
the core of a star larger than the Sun is driven by
a complex carbon–nitrogen–oxygen cycle that
converts hydrogen to helium. In these massive stars’
cores, hot gases rise toward the surface, and cool gases
fall back in a circulatory pattern known as convection.
After depleting its hydrogen—and subsequently its
helium, carbon, and oxygen—the contracting core
of a massive star becomes unstable and implodes while
the other layers explode as a supernova. The imploding
core may first become a neutron star and, later, a pulsar
or black hole.
The cores of
stars are turbulent in a manner analogous to a boiling
kettle, says Livermore astrophysicist Peter Eggleton.
Driven by enormous heat, the material in a core takes
about a month to completely circulate (our Sun accomplishes
it in about two weeks). “One-dimensional simulations
give you an average of what’s going on in the kettle
instead of telling you what’s happening on a second-to-second
basis, so we are forced to make some bold assumptions.”
Eggleton also says that one-dimensional codes cannot
model time-dependent convection in such events as helium
flashes, which occur in the late stages of a red giant
star.
One of the long-standing
issues of astrophysics has been determining the correct
convective core size of stars. Astronomical observations
have suggested that the convection region is larger
than has been assumed since the 19th century. Astronomers
call the situation convective-core overshoot, meaning
that the core probably extends beyond the long-accepted
boundary.
Determining
the exact size of the convective core is of more than
passing interest. If the core is indeed larger than
has been assumed, then stars could be much older than
has been believed, which has profound implications
for how the universe evolved and its real age. “The
When
low-mass stars such as our Sun become red giants, they
grow a helium core. Eventually the helium core ignites
and begins burning to carbon and oxygen. The ignition
begins in a shell that initially expands and drives
a weak shock into and out of the star. The image shows
the velocity contours of the expanding shell in a cutaway
segment of a star in which ignition is beginning. The
red areas represent the highest velocity, corresponding
to the rapidly expanding shell both in front and in
back (barely visible).
|
modeling of convection is one of the weakest points
in our understanding of stellar structure and evolution,”
says Livermore astrophysicist David Dearborn.
The issue over the size of the convection region is
serving as a way to verify and validate the accuracy
of Djehuty. The code development team made convective
core overshoot a priority in part because the fusion
process occurs during the earliest and simplest phase
of stellar evolution—during what is called the
main sequence. The main sequence is shown on a Hertzsprung–Russell
diagram, which plots stars’ temperatures versus
their brightness, thereby showing their evolution.
Two
simulations taken about 8 minutes apart show the changes
inside the core of a star four times the mass of our
Sun. Colors represent relative velocity (increasing
from blue to yellow), and the arrows show the direction
of convective currents.
“Observations
assure us that our best one-dimensional approximations
of convection are flawed,” says Eggleton. “With
Djehuty, we have a three-dimensional code with accurate
physics to determine what exactly happens in the core.
There are big rivers flowing in stars’ cores, and
we want to follow them.”
One simulation
modeled a star early in its evolution, prior to its
joining the main sequence. As expected, it did not show
any convection motions from thermonuclear fusion. Another
simulation studied a massive star that had just reached
the main sequence and so witnessed the onset of convective
motion from fusion. A third simulation looked at a red
giant, a very old star that possesses a large core of
helium. The helium eventually ignites in what is called
a helium flash.
The simulations
suggested that a star’s convective core indeed
exceeds its classical boundary. Additional computationally
intense simulations, each requiring a month of supercomputer
time, will be done this year to model a star’s
convective core at key stages in its lifetime.
The
Hertzsprung–Russell diagram plots the temperatures
of stars versus their brightness and is useful for plotting
their evolution. This diagram follows a star with six
times the mass of our Sun. The star spends most of its
lifetime in the main sequence, characterized by producing
fusion in its inner core. Djehuty simulations are modeling
stars in every phase of their evolution.
|
|
Need
for 3D Codes
Convection
is only one of many stellar phenomena that require a three-dimensional
simulation code for accurate modeling. Other complex phenomena that
astrophysicists have long desired to simulate include the evolution
of elements created in a star, the preexplosion structure of supernovas,
and the physics of binary stars, which comprise nearly half of the
visible mass of the universe.
Dearborn
says that developing a three-dimensional code to realistically model
stars is challenging for even the most accomplished teams of computer
scientists and astrophysicists. Before Djehuty, three-dimensional
stellar models were limited to about 1 million zones. (Computer
simulations divide an object into numerous small cells, or zones,
whose behavior is governed by sets of physics equations. The totality
of the zones, or cells, is called a mesh.) The million zones represent
only modest segments of a star. Moreover, the simplified models
did not incorporate all the physics pertinent to a star’s core
where nuclear energy is produced, and they did not simulate gravity
in a realistic manner. “While the earlier codes are important
starts toward improving our understanding, it is clear that the
solutions to some problems necessitate whole-star modeling,”
Eggleton says.
The
advent of massively parallel computing, wherein computers have hundreds
and even thousands of processors, and Livermore’s participation
in the National Nuclear Security Administration’s Stockpile
Stewardship Program—to assure the safety and reliability of
the nation’s nuclear stockpile—led Livermore scientists
to gain expertise in supercomputers and parallel codes. Along with
astrophysicist Kem Cook, Dearborn and Eggleton saw that Livermore
was becoming a uniquely qualified institution to move the calculation
of stellar properties to a higher level of understanding. In particular,
they saw that one element of stockpile stewardship, which uses massively
parallel computing techniques to simulate the performance of nuclear
warheads and bombs in a program called Advanced Simulation and Computing
(ASCI), would be pertinent to their quest for a whole-star, three-dimensional
model.
Dearborn
and Eggleton’s vision was to take advantage of Livermore’s
expertise in ASCI computations, code and algorithm development for
massively parallel computers, astrophysics, high-energy-density
physical data and processes, and experience in interdisciplinary
coordination to attack the fundamental questions of stellar structure
and evolution.
 |
Increasingly
magnified sections of a star with four times the mass of the
Sun can be seen in these Djehuty simulations. Here, (a) and
(c) are the same as (b) and (d), respectively, but show the
location of mesh zones. A closeup of the star’s convective
core is shown in (e). Colors represent relative velocity (increasing
from blue to yellow). The bulk of motion lies in the core, where
convection currents driven by carbon–nitrogen–oxygen
burning occur. The areas of convection appear to extend beyond
what one-dimensional models depict, but Djehuty’s models
are consistent with recent astronomical observations. (f) A
two-dimensional slice of a Djehuty three-dimensional simulation
depicting convection currents deep inside the core. The arrows
signify the directions of the currents. |
 |
A Laboratory-Wide Team
In
1999, Dearborn and Eggleton assembled a team to develop Djehuty
as a three-year Strategic Initiative under Laboratory Directed Research
and Development funding. The collaboration has included John Castor,
Steven Murray, and Grant Bazan from the Defense and Nuclear Technologies
Directorate; Kem Cook from the Physics and Advanced Technologies
Directorate; Don Dossa and Peter Eltgroth from the Computation Directorate’s
Center for Applied Scientific Computing; and several other contributors.
“Collaboration from throughout the Laboratory has been essential
in this project,” says Dearborn.
The team designed Djehuty to operate on massively parallel machines
with the best available physical data about stars and with algorithms
tailored specifically for the massively parallel environment. Notes
Dearborn, “There’s been tremendous work at the Laboratory
in developing parallel codes and learning how to do calculations
in a manner that won’t bog down the machines.” The code
development process involved assembling and reconfiguring a number
of Livermore codes that already existed, many of them parts of unclassified
software belonging to the ASCI program, and optimizing them for
astrophysical simulations.
Djehuty also takes advantage of the Laboratory’s significant
knowledge about opacity (a measure of the distance photons at a
particular frequency travel through a particular material) and equations
of state (the relationship between a material’s pressure, temperature,
and volume). Opacity and equation of state are two key pieces of
data that are used in stockpile stewardship work for studying matter
under extreme conditions. In that respect, says Dearborn, developing
Djehuty is well aligned with Livermore’s programmatic interests
that focus on understanding high-temperature physics and performing
numerical simulations of complex physical reactions.
The code currently features accurate representations of different
elements’ equations of state, opacities, radiative diffusion
transport (how photons are absorbed and reemitted when they interact
with atoms and electrons in a star’s interior), and nuclear
reaction network (fusion reaction rates and abundance of species
formed). Finally, Djehuty features a gravity package for spherical
stars, a provision that is being improved significantly so it will
be possible to simulate a host of aspherical stellar objects.
 |
Some
postdoctoral scientists and the project leaders on the Djehuty
development team. From left, Rob Cavallo, Stefan Keller, team
leaders Peter Eggleton and David Dearborn, and Sylvain Turcotte.
|
The
First Simulation
The
team’s early strategy was to test the code’s accuracy
and achieve some optimization of it. In September 2000, using the
680-gigaops (billion calculations per second) TeraCluster 2000 (TC2K)
parallel supercomputer at Livermore, the team successfully executed
a three-dimensional simulation of a star. This was the first three-dimensional
simulation of an entire star, but it ran on just one of TC2K’s
512 processors, using only some of the code’s physics on a
modest mesh containing approximately 400,000 cubic zones. “Our
first models were too small to accurately represent a star’s
structure, but they were sufficient to study different zone mesh
structures and to optimize the physics equations we were using,”
says Dearborn.
Satisfied
with the early simulations on one processor, the team then modified
the code to run in a massively parallel computing environment. “It’s
a big transition going from one to many processors because we need
at least 10 million zones to model an entire star,” says Dearborn.
Fortunately, he says, Livermore has invested significant resources
to figure out how to break up a complex physics problem, such as
following fusion reactions in time, for efficient processing by
hundreds and even thousands of processors.
Generating
and monitoring large three-dimensional meshes containing millions
of zones is a huge task. To aid computing, the Djehuty team constructs
a mesh sphere of seven blocks: one in the center and six surrounding
it. The outlying six are distorted at their outer edges to make
them spherical. Each block contains at least 1 million zones. Each
zone represents thousands of kilometers on a side, and several thousand
zones are assigned to a processor. All the processors must communicate
efficiently with each other simultaneously. The key to Djehuty’s
simulation power is its ability to access many processors to efficiently
compute the physics in each of the millions of zones. “We’re
fortunate to have so many people who can develop a code like this,”
says Dearborn.
The
team has run simulations on increasing numbers of processors on
the TC2K. Several simulations, using 128 processors and 56-million-zone
meshes, were some of the largest astrophysics calculations ever
performed; they generated close to a terabyte (trillion bytes) of
data. The team has also begun to perform simulations on Livermore’s
ASCI Frost, the unclassified portion of ASCI White, currently the
world’s most powerful supercomputer. Simulations on ASCI Frost
have used 128 of that machine’s processors to evolve stars
with 60-million-zone meshes.
With the code running satisfactorily
in a massively parallel environment, Dearborn and Eggleton focused
on resolving a long-standing controversy in astrophysics. That controversy
surrounds the discrepancy between the results from one-dimensional
stellar models and data gained from astronomical observations concerning
the size of the convection region inside a star. (See the box above.)
This region is where hot plumes of gas rise and fall. The team has
simulated the cores of several stars, ranging from young stars before
the onset of fusion reactions to old stars about half the age of
the universe. Eggleton says that one-dimensional computer models
are especially incomplete in simulating late stellar evolution,
which is often characterized by deep mixing of gases and sudden
pulses of energy.
Virtual
Telescope at Work
Eggleton
compares Djehuty to a kind of virtual telescope that can take snapshots
during a star’s lifetime of several billion years and examine
in detail the star’s structure and the various physical processes
at play. “There is no comparable three-dimensional code, although
there have been heroic efforts to develop one,” he says. As
a result of the early simulations, the Livermore team anticipates
being able to accurately model in three dimensions, for the first
time, a host of important stellar objects. For example, Djehuty
will be vital to understanding supernovas, the brightest objects
in the universe, and about which much is unknown, as well as Cepheids.
Dearborn
predicts that Djehuty will provide an important link between theory
and observation that will further our knowledge of stellar structure
and evolution. Livermore’s Stefan Keller is conducting a number
of observational studies to verify the Dhejuty simulations. One
study uses a certain population of Cepheids to observationally determine
the relationship between mass and luminosity that is dependent on
the original amount of mixing in the star’s convective core.
Preliminary results indicate that these Cepheids are considerably
more luminous than predicted by standard one-dimensional models,
a result suggesting a larger degree of mixing than was previously
thought. Djehuty simulations appear to confirm the observations.
In another study, astrophysicist Rob Cavallo is observing variations
in the surface abundances of some elements in evolved red giant
stars. The variations are caused by some form of nonconvective mixing
process, which can only be determined with the use of a fully three-dimentional
code such as Djehuty.
The team is also working
to improve the code and better interpret its output. One goal is
improving the accuracy of opacities. “There are a range of
problems where a star’s behavior depends on the opacity of
material whose composition is rapidly changing,” says Dearborn.
The team plans to attack those problems by permitting the code to
generate opacity levels using OPAL, a database of stellar opacity
that was developed at Livermore several years ago. (See S&TR,
April 1999, Duplicating
the Plasmas of Distant Stars.)
Another task is improving the techniques to better visualize and
thereby understand the vast amounts of data generated by Djehuty.
Analysis and visualization are the key for turning huge numerical
simulations into scientific understanding, says Dearborn, and at
present, “We must improve our ability to analyze three-dimensional
structures. With longer, larger, and more realistic simulations,
we must develop better tools to analyze our simulations to extract
the greatest amount of information. We can’t eyeball 10 million
zones in three dimensions. We must have ways for a computer to look
for irregularities and flag them.”
Recently, the team began
using MeshTV, a program that was designed at Livermore to visualize
data for three-dimensional meshes. MeshTV can display an animation
of data changing over time and permit a user to rotate, zoom, or
pan an object while a movie assembled from the data is playing.
(See S&TR, October
2000, The Many
Faces of Carbon Dioxide.)
A
Continual Code Development
Djehuty
development will never be finished, although it will eventually
become much less a development code and more a production code ready
for use. The team continues to enhance Djehuty’s physics and
refine its algorithms. Development is also under way to permit simulation
of rapidly rotating stars and, in particular, binaries. Binary stars
revolve around a common center of gravity and sometimes exchange
some of their mass or even merge into one star. Often, one binary
is distorted by the gravitational pull of the other, and the result
is seen in varying brightness.
“Simulating
binaries has become our main physics priority,” says Dearborn.
“We want to see how mass comes off one star and is absorbed
by the other.” One-dimensional codes don’t work for binaries
because when two stars interact, the problem is three-dimensional.
Binary
simulations require a more accurate means to simulate gravity, one
that automatically changes to reflect a star’s size, shape,
and internal physics. Once this enhanced gravity treatment is incorporated
into Djehuty, the code will be able to represent binaries as well
as stellar objects that are not perfectly spherical. “Once
work on binaries begins,” says Dearborn, “we will enter
completely new territory because calculations so far have been very
crude.”
The
Livermore effort to revolutionize stellar evolution and modeling
calculations has been well received at two international conferences.
The enthusiasm generated by this work has led to two proposals to
the National Aeronautics and Space Administration from U.S. academic
researchers interested in collaborating with the Djehuty team on
binary star evolution. Other researchers have proposed using the
code to study white dwarfs, the phase of stellar evolution that
occurs late in stars’ lifetimes, depending upon their starting
masses. Dearborn and Eggleton have also received inquiries about
the possibility of modifying the code to run simulations of large
planets and brown dwarfs.
Several
postdoctoral scientists and university students have joined the
Djehuty development team. With a user manual recently completed,
the team is seeking university collaborators, both graduate students
and visiting scientists, who would visit for several months at a
time and join in astrophysical research that can be done nowhere
else.
Dearborn
and Eggleton hope to see a user facility established at the Livermore
branch of the University of California’s Institute of Geophysics
and Planetary Physics (IGPP). The Livermore IGPP currently collaborates
with all UC campuses, more than thirty U.S. universities, and more
than twenty international universities. “Djehuty is a unique
institutional asset for attracting astronomers and physicists interested
in stars and what can be learned from them,” says Eggleton.
—Arnie Heller
Key Words: Advanced
Simulation and Computing (ASCI), ASCI Frost, ASCI White, binary
stars, brown dwarfs, Cepheids, convective core, Djehuty, helium
flash, Hertzsprung–Russell diagram, Institute of Geophysics
and Planetary Physics (IGPP), Mesh TV, stellar evolution, supernovas,
TeraCluster 2000 (TC2K), white dwarfs.
For further
information contact David S. Dearborn (925) 422-7219 (dearborn2@llnl.gov).
|
