HTAR Command |
Availability: |
HTAR is installed and supported on the following LLNL platforms:
HTAR is installed, but not supported, on the following platform at SDSC (use at your own risk): | ||||||||||||||||||||||||||||||||||||||||||||||
Purpose: |
The htar command manipulates HPSS-resident tar-format archives by writing files to, or retrieving files from, the High Performance Storage System (HPSS). Files written to HPSS are in the POSIX 1003.1 "tar" format, and may be retrieved from HPSS, and read by native "tar" programs. The local files used by the htar command are represented by the Filespec parameter. If the Filespec parameter refers to a directory, then that directory, and, recursively, all files and directories within it, are referenced as well. Unlike the standard Unix "tar" command, there is no default archive device; the "-f Archive" flag is required. | ||||||||||||||||||||||||||||||||||||||||||||||
Why Use htar? |
htar has been optimized for creation of archive files directly in HPSS, without having to go through the intermediate step of first creating the archive file on local disk storage, and then copying the archive file to HPSS via some other process such as ftp or hsi. The program uses multiple threads and a sophisticated buffering scheme in order to package member files into in-memory buffers, while making use of the high-speed network striping capabilities of HPSS. In most cases, it will be significantly faster to use htar to create a tar file in HPSS than to either create a local tar file and then copy it to HPSS, or to use tar piped into ftp (or hsi) to create the tar file directly in HPSS. In addition, htar creates a separate index file, which contains the names and locations of all of the member files in the archive (tar) file. Individual files and directories in the archive can be randomly retrieved without having to read through the archive file. Because the index file is usually smaller than the archive file, it is possible that the index file may reside in HPSS disk cache even though the archive file has been moved offline to tape. Since htar uses the index file for listing operations, it may be possible to list the contents of the archive file without having to incur the time delays of reading the archive file back onto disk cache from tape. It is also possible to create an index file for a tar file that was not originally created by htar or to recreate an index that has been unintentionally deleted. |
||||||||||||||||||||||||||||||||||||||||||||||
Command Format: |
htar -{c|t|x|X} -f Archive [-?] [-B] [-E] [-L inputlist] [-h] [-m] [-o] [-d debuglevel] [-p] [-v] [-V] [-w] [-I {IndexFile | .suffix}] [-Y [Archive COS ID][:Index File COS ID]] [-S Bufsize] [-T Max Threads] [Filespec | Directory ...] | ||||||||||||||||||||||||||||||||||||||||||||||
Options: |
Two groups of flags exist for the htar command; "action" flags and "optional" flags. Action flags specify the operation to be performed by the htar command, and are specified by one of the following: -c, -t, -x, -X At least one action flag must be selected in order for the htar command to perform any useful function (note: in the initial implementation, one and only one action can be specified per execution).
Also note that the files to be "htarred" [Filespec | Directory] can be specified in one of the following forms: WildcardPath or Pathname or Filename "WildcardPath" is a path specification that includes standard filename pattern-matching characters, as specified for the shell that is being used to invoke htar. The pattern-matching characters are expanded by the shell and passed to htar as command line arguments. Note that using wildcard characters for the -t and -x actions may not work as expected unless there are existing local files that match the pattern. For example, "htar -xf someFile.tar a*" will only extract files beginning with "a" in "someFile.tar" that also already exist in the current local working directory. |
||||||||||||||||||||||||||||||||||||||||||||||
Usage Notes: |
"Archive" and "Member" files HTAR Index File (.idx) It is also possible to create an index file for an archive file that was not created by htar, by using the "Build Index" [-X] function (see description of -X flag above). By default, the index filename is created by adding ".idx" as a suffix to the Archive name specified by the -f parameter. A different suffix or index filename may be specified by the "-I " option, as described above. By default, the Index File is assumed to reside in the same directory as the Archive File. This can be changed by specifying a relative or absolute pathname via the -I option. The Index file's relative pathname is relative to the Archive File directory unless an absolute pathname is specified. Use of Absolute Pathnames HTAR Consistency File /usr/tmp/HTAR_CF_CHK_64474_982644481 This file is used to verify the consistency of the Archive File and the Index File. Unless the file is explicitly specified, HTAR does not extract this file from the Archive when the -x action is selected. The file is listed, however, when the -t action is selected. Tar File Restrictions The prefix buffer can be a maximum of 155 bytes and the name buffer can hold a maximum of 100 bytes. Since some implementations of TAR require the prefix and name buffers to terminate with a null ('\0') character, htar enforces the restriction that the effective prefix buffer length is 154 characters (+ trailing zero byte), and the name buffer length is 99 bytes (+ trailing zero byte). If the path name cannot be split into these two parts by a slash, it cannot be archived. This limitation is due to the structure of the tar archive headers, and must be maintained for compliance with standards and backwards compatibility. In addition, the length of a destination for a hard or symbolic link (the 'link name') cannot exceed 100 bytes (99 characters + zero-byte terminator). HPSS Default Directories Local Temporary Directory HTAR Memory Restrictions Authentication HTAR Execution Environment HTAR_COS - set to the default COS ID for the archive file, or the string "auto" to force automatic COS selection based upon file size hints. This environment variable is overridden by the -Y command line option. HPSS_SERVER_HOST - contains the server hostname and optional port number of the HTAR server. HPSS_HOSTNAME - contains the hostname or IP address of the network interface to which HPSS mover(s) should connect when transferring data. This is overridden by the file specified in the PFTP_CONFIG_FILENAME environment variable. The default interface is the one specified by the "hostname" command. Note that this is often a slow interface, such as the control ethernet on an IBM SP2. HPSS_PATH_ETC - pathname of a local directory containing the HPSS network options file PFTP_CONFIG_FILENAME - pathname of a file containing the list of HPSS network interfaces to be used HTAR also references the following non-HPSS environment variables: TMPDIR - used when creating temporary files HOME - used when searching for the network options file (normally only used by HPSS system administrators). Notes
Exit Status 0 Successful completion. >0 An error occurred. Files
Related Information For HPSS file transfer programs: pftp, nft, hsi File Systems Overview for System Management in AIX Version 4 System Management Guide: Operating System and Devices explains file system types, management, structure, and maintenance. Directory Overview in AIX Version 4 Files Reference explains working with directories and path names. Files Overview in AIX Version 4 System User's Guide: Operating System and Devices provides information on working with files. Bugs and Limitations:
| ||||||||||||||||||||||||||||||||||||||||||||||
Example: |
|
||||||||||||||||||||||||||||||||||||||||||||||
Feedback, Questions, or Problems: |