Data Test Program (dt)

What Is this Program Used For?

dt is a generic data test program used to verify proper operation of peripherals, file systems, device drivers, or any data stream supported by the operating system. In its' simplest mode of operation, dt writes and then verifys its' default data pattern, then displays performance statisics and other test parameters before exiting. Since verification of data is performed, dt can be thought of as a generic diagnostic tool.

dt is conditionalized to run on Tru64 UNIX, HP-UX, SUN, ULTRIX, OSF/MK, QNX, Linux, SCO Unixware, and Windows operating systems. There are two versions of dt for Windows, the previous version which requires the Cygnus Solutions cygwin toolkit, and a native version found under the windows subdirectory (Debug and Release versions).

dt command lines are similar to the dd program, which is popular on most UNIX systems. It contains numerous options to give the user control of various test parameters.

dt has been used to successfully test disks, tapes, serial lines, parallel lines, pipes, and memory mapped files. In fact, dt can be used for any device that allows the standard open, read, write, and close system calls. Special support is necessary for some devices, such as serial lines, for setting up the speed, parity, data bits, etc.

Releases

Within NetApp, dt releases are maintained by the SAN Host Utility team in RTP, namely Tom Masterson, who maintains this Wiki describing the latest versions of dt.

Latest dt Kits:

Latest dt source kit Version 16.17

Unpack the above archives via: "gzip -dc kit-name.tar.gz | tar xvf -"

Prebuilt Executables

Within NetApp, you can find prebuilt dt executables in /usr/local/test/bin/dt*. The dt.stable and dt.latest symbolic links are advised to get the most stable or latest version (latest becomes stable after a couple months).

The Work-In-Progress (WIP) versions of dt are located in my account on the production machines in ~rtmiller/Tools/dt.d-WIP. In this directory, you will find subdirectories for each operating system, for example aix5.3, solaris-sparc, linux2.6-x86, windows/Release, etc.

Documentation:

NetApp dt Overview (PowerPoint).
dt LA Testing (PowerPoint).
dt Use Cases
dt No Progress Details
dt Test Recommendations
dt User's Manual (HTML)
dt User's Manual (MS Word)
dt User's Manual (Acrobat)
dt Program Overview (Acrobat).
dt Program Overview (HTML).
dt Program Overview (PowerPoint).
dt Source Overview (Word).
Please Note: Within NetApp, the released dt can be found here: /usr/software/test/bin/dt
The latest work-in-progress versions are found in: ~rtmiller/Tools/dt.d-WIP/{OSsubdir}
The latest released source is in Perforce here: /u/rtmiller/p4/performance/dt.d-v15.27

Rebuilding dt is fairly easy. After extracting the latest source kit, build dt via these steps:

        % cd linux2.6-x86
        % make -f ../Makefile.linux clean
        % make -f ../Makefile.linux VPATH=..

Latest Changes:

What's New as of September 18th, 2008 (Version 16.01)
What's New as of January 13th, 2007 (Version 15.34)
What's New as of August 14th, 2004 (Version 15.10)
What's New as of March 25th, 2003 (Version 14.28)
What's New as of January 29th, 2001 (Version 14.2)
What's New as of April 2nd, 2000 (Version 13.10)
What's New as of January 12th, 2000 (Version 13.0)
What's New as of November 11th, 1999 (Version 12.10)
What's New as of August 2nd, 1999 (Version 12.3)
What's New as of January 13th, 1999 (Version 11.3)
What's New as of April 8th, 1998 (Version 10.15)
What's New as of January 28th, 1998 (Version 10.12)
What's New as of May 14th, 1997 (Version 10.8)
What's New as of March 30th, 1996 (Version 10.2)
What's New as of February 21st, 1996 (Version 9.4)
What's New as of December 9th, 1995 (Version 9.0)
What's New as of July 27th, 1995 (Version 8.0)

Mail Send mail to admin of this page: Robin.Miller@netapp.com Home To Robin's home page.

Last modified: July 30th, 2009


Uses For dt:

Those people with an imagination can find many uses for dt, but I'll list a few just to whet your appetite: Although I've started to add specific testing of serial lines with modem control, this support is incomplete and untested.

I hope you find dt as useful as I have. This is usually one of the first tools I port to a new operating system, since it's an excellent diagnostic and performance tool.

Help Text:

% dt help
Usage: dt options...

    Where options are:
        if=filename      The input file to read.
        of=filename      The output file to write.
        pf=filename      The data pattern file to use.
        bs=value         The block size to read/write.
        log[tu]=filename The log file name to write.
                         t=truncate, u=unique (w/pid)
        aios=value       Set number of AIO's to queue.
        alarm=time       The keepalive alarm time.
        keepalive=string The keepalive message string.
        pkeepalive=str   The pass keepalive msg string.
        tkeepalive=str   The totals keepalive msg string.
        align=offset     Set offset within page aligned buffer.
    or  align=rotate     Rotate data address through sizeof(ptr).
        capacity=value   Set the device capacity in bytes.
        dispose=mode     Set file dispose to: {delete, keep, or keeponerror}.
        dlimit=value     Set the dump data buffer limit.
        dtype=string     Set the device type being tested.
        idtype=string    Set input device type being tested.
        odtype=string    Set output device type being tested.
        dsize=value      Set the device block (sector) size.
        errors=value     The number of errors to tolerate.
        files=value      Set number of tape files to process.
        flow=type        Set flow to: none, cts_rts, or xon_xoff.
        incr=value       Set number of record bytes to increment.
    or  incr=variable    Enables variable I/O request sizes.
        iodir=direction  Set I/O direction to: {forward or reverse}.
        iomode=mode      Set I/O mode to: {copy, test, or verify}.
        iotype=type      Set I/O type to: {random or sequential}.
        iotpass=value    Set the IOT pattern for specified pass.
        iotseed=value    Set the IOT pattern block seed value.
        history=value    Set the number of history request entries.
        hdsize=value     Set the history data size (bytes to save).
        min=value        Set the minumum record size to transfer.
        max=value        Set the maximum record size to transfer.
        lba=value        Set starting block used w/lbdata option.
        lbs=value        Set logical block size for lbdata option.
        limit=value      The number of bytes to transfer.
        flags=flags      Set open flags:   {excl,sync,...}
        oflags=flags     Set output flags: {append,trunc,...}
        oncerr=action    Set child error action: {abort or continue}.
        noprogt=value    Set the no progress time (in seconds).
        noprogtt=value   Set the no progress trigger time (secs).
        notime=optype    Disable timing of specified operation type.
        parity=string    Set parity to: {even, odd, or none}.
        passes=value     The number of passes to perform.
        pattern=value    The 32 bit hex data pattern to use.
    or  pattern=iot      Use DJ's IOT test pattern.
    or  pattern=incr     Use an incrementing data pattern.
    or  pattern=string   The string to use for the data pattern.
        position=offset  Position to offset before testing.
        prefix=string    The data pattern prefix string.
        procs=value      The number of processes to create.
        ralign=value     The random I/O offset alignment.
        rlimit=value     The random I/O data byte limit.
        rseed=value      The random number generator seed.
        records=value    The number of records to process.
        runtime=time     The number of seconds to execute.
        slice=value      The specific disk slice to test.
        slices=value     The number of disk slices to test.
        skip=value       The number of records to skip past.
        seek=value       The number of records to seek past.
        step=value       The number of bytes seeked after I/O.
        stats=level      The stats level: {brief, full, or none}
        trigger=type     The trigger to execute during errors.
        volumes=value    The number of volumes to process.
        vrecords=value   The record limit for the last volume.
        enable=flag      Enable one or more of the flags below.
        disable=flag     Disable one or more of the flags below.

    Flags to enable/disable:
        aio              POSIX Asynchronous I/O.(Default: disabled)
        cerrors          Report close errors.   (Default: enabled)
        compare          Data comparison.       (Default: enabled)
        coredump         Core dump on errors.   (Default: disabled)
        debug            Debug output.          (Default: disabled)
        Debug            Verbose debug output.  (Default: disabled)
        edebug           End of file debug.     (Default: disabled)
        rdebug           Random debug output.   (Default: disabled)
        tdebug           Timer debug output.    (Default: disabled)
        dump             Dump data buffer.      (Default: enabled)
        eof              EOF/EOM exit status.   (Default: disabled)
        fsalign          File system align.     (Default: disabled)
        funique          Unique output file.    (Default: disabled)
        fsync            Controls file sync'ing.(Default: runtime)
        header           Log file header.       (Default: enabled)
        hdump            History dump.          (Default: disabled)
        htiming          History timing.        (Default: disabled)
        lbdata           Logical block data.    (Default: disabled)
        logpid           Log process ID.        (Default: disabled)
        looponerror      Loop on error.         (Default: disabled)
        microdelay       Microsecond delays.    (Default: disabled)
        mmap             Memory mapped I/O.     (Default: disabled)
        multi            Multiple volumes.      (Default: disabled)
        noprog           No progress check.     (Default: disabled)
        prefill          Prefill read buffer.   (Default: enabled)
        pstats           Per pass statistics.   (Default: enabled)
        raw              Read after write.      (Default: disabled)
        retryDC          Retry data corruptions.(Default: enabled)
        sighup           Hangup signal control. (Default: enabled)
        stats            Display statistics.    (Default: enabled)
        syslog           Log errors to syslog.  (Default: disabled)
        timestamp        Timestamp each block.  (Default: disabled)
        trigargs         Trigger cmd arguments. (Default: enabled)
        unique           Unique pattern.        (Default: enabled)
        verbose          Verbose output.        (Default: enabled)
        verify           Verify data written.   (Default: enabled)

      Example: enable=debug disable=compare,pstats

    Common Open Flags:
        excl (O_EXCL)         Exclusive open. (don't share)
        ndelay (O_NDELAY)     Non-delay open. (don't block)
        nonblock (O_NONBLOCK) Non-blocking open/read/write.
        direct (O_DIRECT)     Direct disk access. (don't cache data).
        fsync (O_FSYNC)       Sync both read/write data with disk file.
        rsync (O_RSYNC)       Synchronize read operations.
        sync (O_SYNC)         Sync updates for data/file attributes.
        large (O_LARGEFILE)   Enable large (64-bit) file system support.

    Output Open Flags:
        append (O_APPEND)     Append data to end of existing file.
        dsync (O_DSYNC)       Sync data to disk during write operations.
        trunc (O_TRUNC)       Truncate an existing file before writing.

    Delays (Values are seconds, unless microdelay enabled):
        cdelay=value     Delay before closing the file.    (Def: 0)
        edelay=value     Delay between multiple passes.    (Def: 0)
        rdelay=value     Delay before reading each record. (Def: 0)
        sdelay=value     Delay before starting the test.   (Def: 0)
        tdelay=value     Delay before child terminates.    (Def: 1)
        wdelay=value     Delay before writing each record. (Def: 0)
        retry_delay=value Delay before retrying operation. (Def: 5)

    Numeric Input:
        For options accepting numeric input, the string may contain any
        combination of the following characters:

        Special Characters:
            w = words (4 bytes)            q = quadwords (8 bytes)
            b = blocks (512 bytes)         k = kilobytes (1024 bytes)
            m = megabytes (1048576 bytes)  p = page size (4096 bytes)
            g = gigabytes (1073741824 bytes)
            t = terabytes (1099511627776 bytes)
            inf or INF = infinity (18446744073709551615 bytes)

        Arithmetic Characters:
            + = addition                   - = subtraction
            * or x = multiplcation         / = division
            % = remainder

        Bitwise Characters:
            ~ = complement of value       >> = shift bits right
           << = shift bits left            & = bitwise 'and' operation
            | = bitwise 'or' operation     ^ = bitwise exclusive 'or'

        The default base for numeric input is decimal, but you can override
        this default by specifying 0x or 0X for hexadecimal conversions, or
        a leading zero '0' for octal conversions.  NOTE: Evaluation is from
        right to left without precedence, and parenthesis are not permitted.

    Keepalive Format Control:
            %b = The bytes read or written.  %B = Total bytes read and written.
            %c = Record count for this pass. %C = Total records for this test.
            %d = The device name.            %D = The real device name.
            %e = The number of errors.       %E = The error limit.
            %f = The files read or written.  %F = Total files read and written.
            %h = The host name.              %H = The full host name.
            %k = The kilobytes this pass.    %K = Total kilobytes for this test.
            %l = Blocks read or written.     %L = Total blocks read and written.
            %m = The megabytes this pass.    %M = Total megabytes for this test.
            %p = The pass count.             %P = The pass limit.
            %r = Records read this pass.     %R = Total records read this test.
            %s = The seconds this pass.      %S = The total seconds this test.
            %t = The pass elapsed time.      %T = The total elapsed time.
            %i = The I/O mode (read/write)   %u = The user (login) name.
            %w = Records written this pass.  %W = Total records written this test.

    Performance Keywords:
            %bps  = The bytes per second.    %lbps = Logical blocks per second.
            %kbps = Kilobytes per second.    %mbps = The megabytes per second.
            %iops = The I/O's per second.    %spio = The seconds per I/O.

      Lowercase means per pass stats, while uppercase means total stats.

      Default: %d Stats: mode %i, blocks %l, %m Mbytes, pass %p/%P, elapsed %t
                     or if pass statistics summary is disabled:
               %d Stats: mode %i, blocks %L, %M Mbytes, pass %p/%P, elapsed %T

    Log File Format Keywords:
            %iodir = The I/O direction.     %iotype = The I/O type.
            %host  = The host name.         %pid    = The process ID.
            %user  = The user name.

      Example: log=dt_%host_%user_%iodir_%iotype-%pid.log

    Pattern String Input:
            \\ = Backslash   \a = Alert (bell)   \b = Backspace
            \f = Formfeed    \n = Newline        \r = Carriage Return
            \t = Tab         \v = Vertical Tab   \e or \E = Escape
            \ddd = Octal Value    \xdd or \Xdd = Hexadecimal Value

    Prefix Format Control:
            %d = The device name.           %D = The real device name.
            %h = The host name.             %H = The full host name.
            %p = The process ID.            %P = The parent PID.
            %u = The user name.

      Example: prefix="%u@%h (pid %p)"

    Time Input:
            d = days (86400 seconds),      h = hours (3600 seconds)
            m = minutes (60 seconds),      s = seconds (the default)

        Arithmetic characters are permitted, and implicit addition is
        performed on strings of the form '1d5h10m30s'.

    Trigger Types:
            br = Execute a bus reset.
            bdr = Execute a bus device reset.
            seek = Issue a seek to the failing lba.
            cmd:string = Execute command with these args:
              string dname op dsize offset position lba errno noprogt
              args following cmd:string get appended to above args.

            The first three options require Scu in your PATH.

    Defaults:
        errors=1, files=0, passes=1, records=0, bs=512, log=stderr
        pattern=0x39c39c39, dispose=delete, align=0 (page aligned)
        aios=8, dlimit=512, oncerr=continue, volumes=0, vrecords=1
        iodir=forward, iomode=test, iotype=sequential, stats=full
        iotseed=0x01010101, hdsize=32

    --> Date: April 20th, 2009, Version: 16.20, Author: Robin T. Miller <--
%