There are a number of documents which the users of XMM-Newton
data should be aware of. These documents include the SSC Products
Specification, Data Files Handbook, Reading Data
Products CD's (the most recent versions of these documents can be
found in the SOC Document section under
http://xmm.esac.esa.int/external/xmm_user_support/documentation/uhb_frame.shtml),
and the SAS Users Guide
(http://xmm.esac.esa.int/external/xmm_user_support/documentation/sas_usg/USG/USG.html).
For observation data sets going to US PIs, the GOF
makes the data available online after PGP encryption
and after converting
the file names to upper case. When the proprietary period for the
observation expires the data are decrypted leaving the file names
unchanged. A simple decryption script, minus the relevant keys of
course, can be found at:
ftp://legacy.gsfc.nasa.gov/xmm/software/decrypt.pl.
NOTE: Laura Brenneman wrote a script and accompanying help file
that gives explicit directions on how to most quickly pull over all
the files in a data set from the archive, as well as decrypting, and
uncompressing the files in preparation for data analysis. This package
can be found at:
ftp://legacy.gsfc.nasa.gov/xmm/software/prepare_xmm_data.tar.gz.
and contains the following files: README, decrpyt.pl, and
prepare_xmm_data.pl.
One of the first steps that should be taken when
examining your data is to check to see what you actually have. XMM-Newton observations can be broken into several exposures which
are each assigned separate observation numbers. These separate
exposures can be radically different in length and can also have the
different instruments in different modes. For example, in one case
the full observation was 60 ks with EPIC and RGS active but there was
one delivered exposure which was ks and had only RGS active.
(This can happen because the RGS can operate farther into regions of
higher radiation than the EPIC detector. The additional observation
time can be considered an additional exposure with only the RGS
active.) Two files are useful for this examination. First, the
primary HTML page is INDEX.HTM which is included in the Pipeline
Products. This page lists basic information for the observation plus
the operational modes, filters, and exposure start and stop times for
the individual instruments. It also has links to various summary
pages, including those for the instruments. (In the case above, the
EPIC summary page simply stated that ``EPIC exposures processed by PPS
None.'') Specifically, LOOK at the P*SUMMAR0000.HTM files
in the pipeline products (easily available through the links).
Second, to quickly access images from the various
instruments, examine the PPSGRA Pipeline Products page, viewable
with a web browser. These files have the nomenclature P*PPSGRA000_0.HTM.
(see § 3.3.3).
Proprietary XMM-Newton data is available for download via your XSA account. Email instructions from the XMM-SOC at Vilspa are sent to the address on record with detailed directions on how to retrieve your data via the XSA.
The data files can be considered to come in two groups in separate subdirectories when retrieved, the Observation Data Files (ODF) files and Pipeline Processing (PPS) files. The ODF data contain all of the observation-specific data necessary for reprocessing the observation. The PPS data contain, among other things, calibrated photon event files and source lists.
For observation data sets going to US PIs, the GSFC GOF makes the data available in two directories containing the following groups of files.
ODF data come with file names in the following format:
NOTE: For SAS processing, the file names should contain all upper case characters. However, at least with early CDs, the file names used lower case characters. The GSFC XMM-Newton GOF provides a script to rename the files.
Data ID | Contents |
EPIC files | |
IME | Event list for individual CCDs, imaging mode |
RIE | Event list for individual CCDs, reduced imaging mode |
CTE | Event list for individual CCDs, compressed timing mode |
TIE | Event list for individual CCDs, timing mode |
BUE | Event list for individual CCDs, burst mode |
AUX | Auxiliary file |
CCX | Counting cycle report (auxiliary file) |
HBH | HBR buffer size, non-periodic housekeeping |
HCH | HBR configuration, non-periodic housekeeping |
HTH | HBR threshold values, non-periodic housekeeping |
PEH | Periodic housekeeping |
PTH | Bright pixel table, non-periodic housekeeping |
DLI | Discarded lines data |
PAH | Additional periodic housekeeping |
PMH | Main periodic housekeeping |
RGS files | |
AUX | on-board processing statistics |
SPE | raw event list for one CCD |
DII | diagnostic images |
D1H | CCD readout settings |
D2H | CCD readout settings |
PFH | housekeeping data |
ODX | pixel offset data |
XMM files | |
ATS | spacecraft attitude history |
OM files | |
IMI | imaging file |
THX | tracking history file |
WDX | window data auxiliary file |
NPH | non-periodic housekeeping file |
PEH | periodic housekeeping file |
PAX | field acquisition data |
RFX | priority reference frame data |
PFX | priority fast mode data |
FAE | event list (if fast mode was used) |
The pipeline processing produces quite a number of useful products which allow a first look at the data, but can overwhelm the user by their sheer numbers. The first place to look is the INDEX.HTM page which organizes the presentation of the data and provides links to other PP pages. The INDEX.HTM page also lists general observation information (target, date, time, etc.) and instrument modes.
The INDEX.HTM page provides links to various observation summary pages, which have names with the following nomenclature:
PP data contain some immediately useful data products such as calibrated photon event lists, source lists, and images. While there are a large number of products which come in a single directory, they can be associated in up to 15 groupings; see Table 3.2. (The number of groups can vary depending on the number of operational instruments, e.g., if the OM is turned off there are no OM products.) Further information on each of these groupings and associated files, such as file contents, file types, and how they may be viewed, can be found in Table 3.3. Each group has an associated HTML file which organizes access to the files and provides a limited description of them. The names of the HTML files are of the following form:
Group ID | Contents |
PP files | |
PPSDAT | Contains the Calibration Index File (CIF) used in the pipeline processing (*CALIND*), |
PPS information, and the attitude history time series (*ATTTSR*) in gzipped FITS | |
or ASCII format. | |
PPSGRA | Contains the OM tracking history plots, PPS, EPIC, OM, RGS observation, and PPS |
run summaries. NOTE: CHECK THESE OUT | |
PPSMSG | ASCII file containing pipeline processing report |
EPIC files | |
CRSCOR | Contains PDF files of POSS II finding charts, HTML files of cross correlations |
with the SIMBAD data base, FITS tables for the detected sources | |
EANCIL | Contains the exposure maps in a variety of energy bands and the source-detection |
sensitivity maps for the EPIC instruments. The sensitivities are in units of | |
counts s![]() |
|
in the FITS header. The files are gzipped with a .FTZ extension. | |
EEVLIS | Contains calibrated photon event files for the EPIC detectors. If the files are |
sufficiently large they may be separated into two tar files. The files are gzipped | |
fits files with a .FTZ extension. | |
ESKYIM | This group contains the event images in a variety of energy bands. The fits files |
are gzipped with a .FTZ extension, the full images also come as PNG images. | |
ESRLIS | Contains EPIC observation source lists. There is an HTML page of the merged |
source list and gzipped fits tables of source lists from the different instruments | |
and source detection tasks. | |
OM files | |
OIMAGE | Contains OM sky images in gzipped FITS format. |
OMSLIS | Contains OM observation source lists in gzipped FITS format. |
OMSRTS | Contains OM star tracking time series in gzipped FITS format. |
RGS files | |
REVLIS | Contains the RGS source and event lists in gzipped FITS format |
REXPIM | Contains the RGS exposure maps in gzipped FITS format |
RIMAGE | Contains the RGS images (both energy dispersion and cross dispersion) in gzipped |
FITS and PNG formats | |
RSPECT | Contains the RGS source and background spectra in gzipped FITS and PDF formats |
The data file names are of the form (see Table 41 in the XMM Data Files Handbook):
Group ID | File ID | Contents | File Type | View With |
EPIC files | ||||
CRSCOR | FCHART | Finding chart | Acrobat Reader | |
ROSIMG | ROSAT image of region | Acrobat Reader | ||
SNNNNN![]() |
Source cross-correlation results | Zipped FITS | fv | |
DNNNNN![]() |
Catalog descriptions | Acrobat Reader | ||
FNNNNN![]() |
FOV cross-correlation result | Zipped FITS | fv | |
ESKYIM | IMAGE_8 | Sky image 0.2 - 12.0 keV | Zipped FITS | ds9, Ximage, fv |
IMAGE_1 | Sky image 0.2 - 0.5 keV | Zipped FITS | ds9, Ximage, fv | |
IMAGE_2 | Sky image 0.5 - 2.0 keV | Zipped FITS | ds9, Ximage, fv | |
IMAGE_3 | Sky image 2.0 - 4.5 keV | Zipped FITS | ds9, Ximage, fv | |
IMAGE_4 | Sky image 4.5 - 7.5 keV | Zipped FITS | ds9, Ximage, fv | |
IMAGE_5 | Sky image 7.5 - 12.0 keV | Zipped FITS | ds9, Ximage, fv | |
EANCIL | EXPMAP_8 | Exposure map 0.2 - 12.0 keV | Zipped FITS, PNG | ds9, Ximage, fv, web browser |
EXPMAP_1 | Exposure map 0.2 - 0.5 keV | Zipped FITS | ds9, Ximage, fv | |
EXPMAP_2 | Exposure map 0.5 - 2.0 keV | Zipped FITS | ds9, Ximage, fv | |
EXPMAP_3 | Exposure map 2.0 - 4.5 keV | Zipped FITS | ds9, Ximage, fv | |
EXPMAP_4 | Exposure map 4.5 - 7.5 keV | Zipped FITS | ds9, Ximage, fv | |
EXPMAP_5 | Exposure map 7.5 - 12.0 keV | Zipped FITS | ds9, Ximage, fv | |
EEVLIS![]() |
MIEVLI | MOS imaging mode event list | Zipped FITS | xmmselect, fv, Xselect |
PIEVLI | PN imaging mode event list | Zipped FITS | xmmselect, fv, Xselect | |
TIEVLI | PN, MOS timing mode event list | Zipped FITS | xmmselect, fv, Xselect | |
ESRLIS | EBLSLI | Box-local detect source list | Zipped FITS | fv |
EBMSLI | Box-map detect source list | Zipped FITS | fv | |
EMSRLI | Max-like detect source list | Zipped FITS | fv | |
OBSMLI | Summary source list | Zipped FITS, HTML | fv, web browser | |
RGS files | ||||
REVLIS | SRCLI_ | RGS Source Lists | Zipped FITS | fv |
EVENLI | RGS Event lists | Zipped FITS | xmmselect, fv | |
REXPIM | EXPMAP | RGS Exposure Maps | Zipped FITS | ds9, Ximage, fv |
RSPECT | SRSPEC1 | 1st Order Source Spectra | Zipped FITS | Xspec, fv |
SRSPEC2 | 1st Order Source Spectra | Zipped FITS | Xspec, fv | |
BGSPEC1 | 1st Order Source Spectra | Zipped FITS | Xspec, fv | |
BGSPEC2 | 1st Order Source Spectra | Zipped FITS | Xspec, fv | |
SRSPEC | Spectra Plots | PDF format | Acrobat Reader | |
RIMAGE | ORDIMG | Images, disp. vs. X-disp | Zipped FITS, PNG | ds9, Ximage, fv, web browser |
IMAGE_ | Images, disp. vs. PI | Zipped FITS, PNG | ds9, Ximage, fv, web browser | |
OM files | ||||
OIMAGE | SIMAGE | OM Sky Image | Gzipped FITS | ds9, Ximage, fv |
OMSLIS | SWSRLI | OM Source Lists | Zipped FITS | fv |
OMSRTS | TSTRTS | Tracing Star Time Series | Zipped FITS | fv |