Comments on the TIDE processing software: Last updated (05/18/07). ------------------------------------------------------------------------------- The latest version of the TIDE processing software is tide_lz_v6.1.3. There are two types of the processing code, one runs without IDL and the other runs from IDL. I use the non-idl version for development, testing, and non-routine processing. There is no reason why the IDL version can't be used to do this. The IDL version of the code is in the most recent directory of the exlib account, tide_dir/tide_lz_vn.n.n. The current development account is /export/data/tide/exlib/new/tide_lz_v6.2. Note: not changes have been made to any of the code so it should be identical to 6.1.3. The IDL version of the software is used for all the routine processing except the creation of the high-time resolution (htr) cdf files. It is also the software that users can download and that the web uses for data processing. The only reason that the htr cdf files are created using the non-idl version is to avoid tying up an IDL license to process a lot of data. If necessary, the crontab script used to create the htr cdf files can be modified to use the IDL version. Assosciated with each version of the software is lz_parm. It is an interactive program that determines the type of processing done. It asks for responses to a series of prompts that determine what type of processing is to be done. It is part of the input to the processing program. The only difference in the software for the non-idl and the idl version of the software are some changes in tide_lz_main.c At the top of the tide_lz_main.c, just after the comments is a typedef structure that varies with the version of IDL in use. For the non-idl version, both need to be commented out. For the software run on goewin, the typedef for early versions of IDL needs to be commented out. Do a search on standalone. This block of code needs to be commented out for the IDL version. Do a search on 'IDL Version'. There are 4 blocks of code that start with this phraise in at comment at the beginning and end for the block. For the non-idl version of the code, these need to be comment out. To comment out a block of code have '/*' on the first line and '*/' as the last two characters of the last line of block. Do not have these two sequences any other place in the block. To make sure that a block of code gets used (not commented out), make sure that the first two characters of the comment at the top of the block and the comment at the bottom of the block (if there is one) are '/*' and the last two characters are '*/'. Hopefully, the software is internally documented well enough so that you can find the area that needs to be edited. There is a file called tide_lz.smry in this directory that is old but it has some information in it about the files that make up the processing code. The command to compile either version of the software is simply 'make'. The executable for the non-idl code is tide_lz. For the idl version, make creates a unix software library, tide_lz.so. The command to compile lz_parm is 'make -f Mparm'. ------------------------------------------------------------------------------- Releasing a new version of the software: Before releasing a new version of the of the software, make sure to update the version number in tide_lz_main.c and to add comments to the top of that file that summarize any changes made. Copy the new tide_lz file to /export/data2/tide/db/exe so that it can be used to create the htr cdf files. In exlib, copy the tar_the_files script from the previous version, edit it to change the name of the tar file to the new version and to add any new software/data files not listed. The create the tar file by simply typing 'tar_the_files'. Copy tide_lz.so and lz_parm to /export/data2/tide/db/exe for use by the production software. Also copy any new input data files and plot routines. ftp or secure copy the tar file to web on satyr and psloan on goewin. On satyr in /home/web/htdocs/TIDE/software and on goewin in /space/www1/tide/software, put a copy of the tar file for the new version and then edit index.html and change the name of the current version of the software. Then on satyr in /home/web/httpd_1.5.2/cgi-bin/TIDE/lz_graphics-dir and on goewin in /space/cgi-bin/tide/lz_graphics-dir, create a directory for the new version, copy the tar file to it, and untar the file (tar xvf tide_lz_vn.n.n). On goewin edit tide_lz_main.c, search on typedef, and comment out the < idl 5.5 block of code and uncomment the >= idl 5.5 code. Create both tide_lz.so and lz_parm using 'make' and 'make -f Mparm'. Copy everything in the new directory to tide_lz_current. If 'cp * ../tide_lz_current' does not work, the try issuing the following commands: cd ../tide_lz_current /bin/rm * cp ../tide_lz_vn.n.n/* . These steps make the latest version of the software accessible from the web interface software. ------------------------------------------------------------------------------- Running the software: To run either program, you must first create a parameter file using lz_parm. This file should have the .parm extension so its easy to keep track of. Once created, .parm files can be used again as long as the desired processing doesn't change. You also need to create an input file with the start and end times of the data to be processed and the location of the input files. The format of this file is slightly different for each version. non-idl version input file format without a day crossing: mm/dd/yy hh:mm:ss mm/dd/yy hh:mm:ss 1 /archive/po_tid_lz_yymm/data/po/tid/lz/yymmddvv.dat 2 /archive/po_tid_lz_yymm/data/po/pre/or/yymmddvv.cdf 3 /archive/po_tid_lz_yymm/data/po/pre/at/yymmddvv.cdf 4 /archive/data/mask/tyymmdd_v2.mask 5 /archive/data/scpot/yyyymmxx.dat idl version input file format without a day crossing: mm/dd/yy hh:mm:ss mm/dd/yy hh:mm:ss parameter filename wpg 1 directory extension 1 /archive/po_tid_lz_yymm/data/po/tid/lz/yymmddvv.dat 2 /archive/po_tid_lz_yymm/data/po/pre/or/yymmddvv.cdf 3 /archive/po_tid_lz_yymm/data/po/pre/at/yymmddvv.cdf 4 /archive/data/mask/tyymmdd_v2.mask 5 /archive/data/scpot/yyyymmxx.dat The first line is the start time, the second line is the stop time. The last five lines are the names of the input files. If the time crosses a day then, include both files one after the other. If the times cross a month, then two spacecraft potential files are required. The first data file line (starts with 1) is the level-zero file, '2' is the orbit file, '3' is the attitude file, '4' is the mask file (can be left blank if the mask is not used), '5' is the spacecraft protential file (can be left off if not used). There are three other files that can be used '6' is the peaks range file, '7' is the mfe b-field file that is used instead of the b-field data in the tide telemetry, and '9' which is the value of the mask correction file. It overrides what is in the parm file. The third through 7th lines of the idl input file are the parameter filename, what type of plot files to create, the color table number, the path location where the output goes (leave blank for current directory), and a file extension to change the standard filename (can be left blank). See run_tide.input.template for more details. To run the non-idl version, type 'tide_lz'. You will be asked to enter the parameter filename and the input filename. Use the *multi idl command files to plot results. To run the idl version, get into idl and typy @run_tide. You will be asked to enter the name of the input file. All plotting is automatic. Use the 'Find files' link on the web pages to find out the name and location of the input files. The web interface takes care of creating both the input and parameter files. ------------------------------------------------------------------------------- Extended studies: Barry has set up a routine on the tide account in the production software directory, /export/data2/tide/db/exe. A script called ksh_cr_batch_mas is set up to run multiple sets of data for a specific processing option. The output files (plots and data files) are created in /export/data3/tide/tide_plots/batch_plots. The command line input for the script is the name of the file that contains the list of start and stop times and the name of the parameter file to be used in the processing. The parameter file is created with lz_parm. ------------------------------------------------------------------------------- Areas that still need to be addressed: The data has not been calibrated since the Stops MCP voltages were change in October 2004. Once the software has been changed, the summary plots and htr cdf files for that time from October 2004 to the point where the change was made probably should be redone. They are currently being done without calibration.