TIDE Level-Zero Processing Software Introduction TIDE Software Meeting - October 31, 1996 Updated - July 28, 1997 TIDE Level-Zero Input Datafiles MSFC-generated Parameter and List Files: MSFC-generated Mass/channel and Instrument Calibration Data MSFC-generated ion count mask files MSFC-generated sector sensitivity EFI spacecraft potential files GSFC-generated Common Data Format orbit and attitude files GSFC-generated TIDE Level-Zero Data files Data Product Description Collapse Option Description TIDE Level-Zero Processing software Location C Header Files tide_lz_def.h tide_lz_glob.h tide_lz_parm.h tide_lz_func.h tide_lz_stru_def.h tide_lz_stru_glb.h tide_lz_stru_ext.h C Source Code Summary lz_parm.c process_parm.c tide_lz_main.c init_processing.c get_lz.c get_spins.c get_dp.c ricedcmp.c process_mass.c mass_conv.c TIDE level-zero output file extensions TIDE Level-Zero plotting software: chromo_summary.pro graybar.pro load_color_cg.pro rgb_wheel.pro tide_lz_call.pro tide_lz_prompt.pro load_color.pro color_bar.pro esse_summary.pro htr_spect.pro spect_summary.pro moments_summary.pro psd_summary.pro plot_gsm_orbit.pro vel_dist_2_para.pro vel_dist_movie.pro vel_dist_perp_para.pro vel_dist_survey.pro Standalone Procedures - used to monitor TIDE/PSI health and safety de_spect.pro de_sum_spect.pro mass_avg_main.pro mass_avg_util.pro mirror_ratios.pro plot_mask.pro psi_hk_smry.pro tide_hk_smry.pro TIDE Level-Zero Input Datafiles ------------------------------- MSFC-generated Parameter and List Files: The parameter file is a binary structure that tells the level-zero processing software what data to retrieve and either plot or save in a file. If the list file is used from IDL, it contains the processing start and stop times, the parameter filename, plot types (window,gif,ps), directory where the plot files are to be placed, a process ID, and the names of the level-zero, orbit, attitude, ion mask, spacecraft potential, peaks location, and/or mfe magnetic field files used in the processing. If the list file is used in the standalone tide_lz, it contains the processing start and stop times and the names of the level-zero, orbit, attitude, ion mask, spacecraft potential, peaks location, and/or mfe magnetic field files used in the processing. MSFC-generated Mass/Channel and Instrument Calibration Data: The mass description and mass/channel calibration data are in a text file identifying each mass by amu, m/q, TOF bin boundaries, and mass label plus it has the channel-depended mass peak corrections as a function of mass and time. The file is call mass_calib.vn, where n is the current version. The instrument calibration data is a text file containing the mirror_ratio and channel-dependent geometric factor and E/Vrpa for ion data prior to 12/7/96 and Stops data after 12/7/96. Data is based on flight calibration. The file is call tide_calib.vn, where n is the current version. MSFC-generated ion count mask files: The sun pulse / noise level masks are binary files containing energy, spin, channel-dependent mass correction values. These files are created every other day from data collected near apogee background checks were made. Mask files for the entire mission are available on the web in http://satyr.msfc.nasa.gov/tmpmo2/mask/ MSFC-generated sector sensitivity: The sector sensitivity file, sector_sens.vn, contains sensitivity corrections for specific dates. If the dates are not in the file and the user wants a correction applied, the values are calculated from a series of equations in the C function, calc_sector_sens in init_processing.c EFI spacecraft potential files The EFI spacecraft potential data are in large month-long text files with records very spin (6 seconds). The files are available on the web in http://satyr.msfc.nasa.gov/tmpmo2/scpot/ GSFC-generated Common Data Format orbit and attitude files: The orbit data are in day long CDF files, one record every minute. The attitude data are in day long CDF files, one record every 10 minutes. GSFC-generated TIDE Level-Zero Data files: Each level-zero file contains one days worth of telemetry data, ~9392 records. All TIDE records are 5800 bytes long in science and maneuver modes. The first record in each file is a header record. It is followed by a level-zero record every 9.2 seconds. Each record has a 300 byte header followed by a major frame. There are 250 minor frames per major frame. The TIDE level-zero consists of 22 bytes of data per minor frame. The first 3 bytes of each minor frame contain the spacecraft housekeeping. All but the 49 bytes assigned to TIDE are zeroed. The 4th byte in the first 142 minor frames is skipped, with the remaining 18 containing TIDE science data. All 19 bytes of the last 108 minor frames contain TIDE science data. The TIDE Science Data, 4608 bytes/major frame, is received in spin packets reported every 6 seconds, ~14400 spins per day. A spin packet includes a 170-byte spin header followed by a series of data products. Each spin packet starts with a unique pair of sync-bytes. Each spin header ends in a checksum. Each data product starts with an 7-byte packet header including a sync byte and ends with a checksum. The number of data products per spin packet varies with the instrument mode and the amount of Rice compression. The are 190 different data products: dp id data product description ------- ---------------------------------------------------------------- 0-41 collapse option 0, one dp for each mass and sector, 31x32 cnts 42-83 collapse option 1, one dp for each mass and sector, 16x32 cnts 84-107 collapse option 2, dp size and content varies 108-131 collapse option 3, dp size and content varies 132-155 collapse option 4, dp size and content varies 156-179 collapse option 5, dp size and content varies 200 collapse option 6, ion count rate, calibration mode 201 collapse option 6, starts,stops,start converts,timeouts,resets 218-220 mass look-up table, instrument mode processor, and data processor memory dumps 221,222 instrument parameters (three different 222 data products) 233 sector 0 direct-events ------- ---------------------------------------------------------------- TIDE Level-Zero Processing software: Use the software downlosd page to get the latest version of the software. C Header Files: tide_lz_def.h contains the variable definitions used in the TIDE level-zero processing software. tide_lz_glob.h contains the global variables used in the TIDE level-zero processing software including mass and image arrays, the 8-bit to 16-bit decompression array, and the location of housekeeping bytes within a major frame. The file contains a description of each variable. tide_lz_parm.h makes all the global variables external (available to any function in any source file) tide_lz_func.h makes all TIDE C functions external tide_lz_stru_def.h contains the data structure definitions used to process TIDE level-zero data, tide_lz_stru_glb.h makes the structures global, and tide_lz_stru_ext.h makes the structures external. The structure definitions are a good source of data descriptions. Some of the are: structure/union variable description ------------------- ------------- --------------------------------- four_bytes mdata byte swapping union File_Label_Record f_label level-zero file label record Hk hk,hk_de[2] housekeeping data hk_save[16] Spin_Packet_Header sp, sp_ion, spin packet header sp_save[16] Data_Product_Header dp data product Eff_Num eff_num time_of_flight efficiency numbers ip ip_now,ip_now instrument parameter data product ip_save[16] Ion ions[100] ion identification Input_Parameters pint processing input parameters Plot_Parameters pprm plot parameters DB_Spin db1,db[16] spin database table DB_HK db_hk housekeeping database table ------------------- ------------- --------------------------------- C Source Code Summary: lz_parm.c is the driver program used to create a TIDE level-zero processing parameter file, it calls init_pint and get_process_parm. process_parm.c contains the functions used to create the TIDE level-zero processing parameter file including: chromo_option determines parameters used to create TIDE chromograms, it calls flux_units flux_units determines the output units and the flux range get_process_parm prompts the user for information necessary to deteremine the main TIDE level-zero processing options, it calls: chromo_option, papco_option, and spect_option init_pint initializes the processing option structure papco_option determines the parameters used to create the moments and high time resolution spectrogram input for the Panel Plot Composer software, it calls flux_units spect_option determines parameters used to create TIDE spectrograms, it calls flux_units tide_lz_main.c is the main program that controls how the TIDE level-zero data is processed. All changes made to the software are documented as comments in this file. It calls init_processing, read_mass_desc, read_calibration, calc_sector_sens, get_file_label, get_lz_rec, process_data, and putline init_processing.c contains the functions used to set up TIDE level-zero processing, initializes variables, and creates output files, routines include: calc_sector_sens reads or calculates sector sensitivity corrections init_ip initializes instrument parameter structure init_masses initializes ion count (masses) array init_plt_prm initializes the plot parameter data structure init_processing controls processing initialization, it calls init_spin, init_ip, init_plt_prm, set_output, and putline init_spin initializes the spin header structure process_data starts and controls the processing of level-zero data, it calls get_lz_rec, process_masses, get_spins, save_de_spect, and putline putline adds comments to the message file (.err) read_mass_desc reads the mass description and mass/channel calibration file and puts it into a data structure set_output creates output files based on the content of parameter file, it calls calc_polar_az_trig_func and save_full_orbit get_lz.c contains the functions used to read TIDE level-zero records including: get_file_label reads and processes the first record, the file label record, in a level-zero data file. get_lz_rec reads a level-zero record, it calls get_file_label, get_mf, and putline get_mf gets housekeeping and major frame bytes from a level-zero record, it calls hk_conv, hk_print, and putline get_spins.c contains the functions used to process TIDE spin data calc_temp converts TIDE housekeeping bytes to temperatures using one of three 5th degree polynomials get_dp_packet gets and processes the next data product, it calls get_lz_rec, putline, get_xpctd_sz, rice_decomp, save_masses, what_mass, get_instr_parms, prnt_instr_parms, move_sing_bytes, and log_singles, get_spins the main function used to get and process TIDE spin packets, it calls get_spin_header, init_masses, get_dp_packet, and save_moments get_spin_header gets the next spin header, it calls get_lz_rec, putline, get_spin_time, save_db_spin_stru, save_de_spect get_spin_time calculates the time in millisecs at the end of a spin and sets up the various time/date labels hk_conv fills in the housekeeping (hk) structure and saves hk summary plot data, it calls calc_temp hk_print creates the TIDE engineering data text display save_db_spin_stru saves spin information in the database parameter structure save_de_spect saves the direct-event color spectrograms, it calls get_orbit_data save_moments saves spin header centroid data and associated mass and instrument parameter data in a text file set_labels determines the ion species and energy labels used by the TIDE plot routines get_dp.c contains the functions used to process TIDE data products get_instr_parms fills in the instrument parameter data structure, the function is a modification of software provided by SwRI log_singles converts calibration mode singles event data from 8-bit to 16-bit integers, logs the results, and saves them for plotting move_sing_bytes puts the calibration mode singles event 8-bit words into their correct array locations prnt_instr_parms creates the instrument parameter text display save_ion_cnts saves ion count rates for selected masses and times in a large text file, one file per channel save_mass_avg averages ion count rates over energy and spin angle and saves them as a function of time save_mass_sum sums counts over energy and spin for a user-specfied time range save_masses uncollapses the ion count rate data products process_mass.c contains the functions used to process TIDE science mode mass data (collapse options 0 - 5), including: chromo_energy calculates energy based TIDE chromograms, it calls change_units chromo_polar calculates polar angle based TIDE chromograms, it calls change_units chromo_spin calculates spin angle based TIDE chromograms, it calls change_units create_image creates and saves spin-averaged flux images, it calls calc_flux, get_oa_data, find_ram, set_labels, chromo_polar, chromo_spin, chromo_energy, spect_data, get_orbit_data, htr_rotated_ions, putline find_ram uses spacecraft velocity (gse) and declination and right ascension to calculate ram velocity in the spacecraft fixed coordinate system flight_calibration calculates ion count sums over spin angle for each energy and polar angle when the mirror steps down get_oa_data reads orbit and attitude CDF files and extracts information necessary to calculate the velocity coordinate transformations get_orbit_data reads orbit files and extracts plot label information get_scpot_orbit reads the spacecraft potential from the monthly EFI files, it calls is_between, compare_time_to, and time_diff htr_rotated_ions fills in data missing as a result of rotated ions in the high time resolution spectrograms process_masses controls the processing of mass data, it calls get_spins, get_scpot_data, save_mass_avg, save_mass_sum, save_ion_cnts, psd_distribution, flight_calibration, save_de_spect, create_image, and putline psd_distribution creates velocity distribution images parallel and/or prependicular to the magnetic field, it calls calc_flux, moments_calc, get_oa_data, and find_ram save_full_orbit saves orbit data for a 17.5 hour time period for spectrogram and velocity distribution plots spect_data calculates energy / spin angle mass spectrograms, it calls change_units what_mass determines what masses were reported based on the data product id number is_between, three functions used in get_scpot_data compare_time_to, time_diff mass_conv.c contains the functions used to change mass units, correct masses, calculate moments, and convert coordinate systems including: calc_flux calculates mirror ratio calibration data, corrects ion counts, and converts counts to user-specified flux units calc_polar_az_trig_func calculates the cosines and sines of the polar and spin angle bins for use in calculating moments change_units converts ion counts to hertz moments_calc calculates plasma density, velocity, and temperature data, it calls get_oa_data and find_ram read_calibration reads the instrument calibration data file ricedcmp.c contains the routines provided by SWRI to decompress the Rice compressed ion count rate data products including: get_bit get_multi_bits rice_decomp get_xpctd_sz dcompnocode dcompfs dcompcfs dcompcfs_bar dcomphigh dcomplow decompress TIDE level-zero output file extensions -------------------------------------- The output created by tide_lz will vary depending on the processing done. The format for the filename is either tyymmddh1m1_h2m2.ext or tyymmddh1m1_h2m2_mm.ext where yymmdd is the start date, h1m1 is the start time, and h2m2 is the stop time. If the file contains data for a single mass, mm is the mass to charge ratio of the ion or it is 00 for Singles or sp for Stops. The extention is a brief indication of the content of the file. Possible extensions include: ext file type -------- ------------------------------------------------------------------- err message/error file - contains information about the processing esp chromogram - energy on horizontal axis, spin angle on vertical axis, color based on polar angle, each panel is a mass sep chromogram - spin angle on horizontal axis, energy on vertical axis, color based on polar angle, each panel is a mass pes chromogram - polar angle on horizontal axis, energy on vertical axis, color based on spin angle, each panel is a mass pse chromogram - polar angle on horizontal axis, spin angle on vertical axis, color based on energy, each panel is a mass espm spectrogram - energy on horizontal axis, spin angle on vertical axis, the panels are polar angle channels, one mass per page esmp spectrogram - energy on horizontal axis, spin angle on vertical axis, the panels are masses, one polar angle per page sepm spectrogram - spin angle on horizontal axis, energy on vertical axis, the panels are polar angle channels, one mass per page semp spectrogram - spin angle on horizontal axis, energy on vertical axis, the panels are masses, one polar angle per page esse spectrogram - energy/spin and spin/energy panels for Stops plus detailed orbit information ehtr high time resolution spectrograms - counts summed over spin and polar angle and display as a function of energy and time, each panel is a mass shtr high time resolution spectrograms - counts summed over energy and polar angle and display as a function of spin angle and time, each panel is a mass phtr high time resolution spectrograms - counts summed over energy and spin angle and display as a function of polar angle and time, each panel is a mass esphtr high time resolution spectrograms - one mass per page, panels are all three types of htr spectrograms eshtr high time resolution spectrograms - one mass per page, panels are energy and spin angle htr spectrograms mom moments text file psd phase space density velocity distribution plots psd_txt text file containing vx, vy, vz, and phase space density values for each energy / spin angle / polar angle count eng TIDE engineering data text display ip TIDE instrument parameters text display mir_rat text file containing TIDE mirror ratios bfld magnetic azimuth and elevation data hk TIDE/PSI housekeeping bytes hk_tid TIDE housekeeping data hk_psi PSI housekeeping data sg_de direct-event spectrogram de_sum summed direct-event summary spectrogram de_chn direct-event text file for channel n mss_avg ion counts summed over energy and spin angle mss_sum ion counts summed over energy and both spin and polar angle mss_chn ion count rates for channel n db_spn database spin table information db_hk database housekeeping table information dmp telemetry byte dumps TIDE Level-Zero plotting software --------------------------------- tide_lz_call.pro two main control procedures tide_lz_prompt.pro chromo_summary.pro procedures used to create chromograms graybar.pro load_color_cg.pro rgb_wheel.pro moments_summary.pro plots moments load_color.pro procedures used by spectrogram and velocity color_bar.pro distribution routines esse_summary.pro procedures used to plot mass spectrograms htr_spect.pro spect_summary.pro psd_summary.pro procedures used to create velocity distribution plots plot_gsm_orbit.pro vel_dist_2_para.pro vel_dist_movie.pro vel_dist_perp_para.pro vel_dist_survey.pro Standalone Procedures - used to monitor TIDE/PSI health and safety de_spect.pro creates daily direct-event spectrograms de_sum_spect.pro creates monthly direct-event spectrograms mass_avg a command file that runs mass_avg_main.pro and mass_avg_util.pro to create the count rate line plots averaged over energy and spin. mirror_ratios.pro creates a plot of mirror ratio as a function of time and energy step plot_mask.pro creates 3-D surface plot of ion mask for a given mass and channel psi_hk_smry.pro creates PSI housekeeping daily summary plots tide_hk_smry.pro creates TIDE housekeeping daily summary plots