Before running DRM:

Berkeley DRM

version 1.2.9

Instructions and trouble shooting guide

11 November 2005

Junmin Gu, Vijaya Natarajan

Arie Shoshani, Alex Sim

Lawrence Berkeley National Laboratory

srm@lbl.gov

http://sdm.lbl.gov/srm-dist

Introduction:

Berkeley DRM version 1.2.x is distributed in VDT as a static binary form.

It consists of the following binaries:

- C++:

o drmServer.linux,

o drm-copy.linux,

o drm-get.linux,

o drm-ls.linux,

o drm-ping.linux,

o drm-status.linux,

o drm-release.linux,

o drm-remove.linux,

o drm-put.linux,

o nameserv.

- Java components (for Web Service access):

o DRM-WSG (DRM-Web Services Gateway)

§ drm-wsg-run

o DRM-GWS (DRM-Gateway to Web Services)

§ drm-gws-run

Notes:

· Fonts in green are what you can customize.

· Fonts in orange are DRM command presented here.

· Fonts in blue are outputs from the commands.

Brief guide on using URL protocol

1. To specify a file in remote gridftp server:

· Format: gsiftp://remotehost/filepath

· For example:
”gsiftp://dmx01.lbl.gov/tmp/test.file”

2. To specify a file in SRM in general:

· Formata: srm://host:port/SRM_Service_Handle?SFN=site_file_name

3. To specify a file in a MSS via DRM in general:

· Format: srm://nameserver_host:nameserver_port/your-file-path
Where the your-file-path is the path on the DRM managed storage space.

· Format: srm://nameserver_host:nameserver_port/your-file-path?remoteobj=remote-corba-obj-name &msshost=your-mss-host&mssport=your_mss_port
Where the order of msshost, mssport and remoteobj is interchangeable.

· For example: “srm://dmx01.lbl.gov:4000/nersc/users/junmin/small_testfile?remoteobj=HRMServerLBNL&msshost=garchive.nersc.gov”
identifies a MSS file with path /nersc/users/junmin/small_testfile from an SRM whose naming service is running at dmx01.lbl.gov:4000, and the corba object name is “HRMServerLBNL”, and the MSS host is “garchive.nersc.gov”.

Chapter 1. DRM and DataMover

Instructions:

1. Preparation before running DRM:
Making sure the following software information is in your local environment

· DRM_HOME where all the DRM software reside

o PATH to include $DRM_HOME

· JAVA_HOME where JDK 1.4.2_07 resides

o PATH/LD_LIBRARY_PATH to reflect appropriate JDK

2. Running DRM in VDT

1) Run CORBA naming service on your machine:

· “$DRM_HOME/nameserv –OAport 4000 &”
(You can use other ports than 4000.)

2) Create your drm.rc file in $DRM_HOME/conf. You can copy the sample, please change these entries to reflect your usage:

· common*NSHost=<the hostname you ran your nameserv in step 1)>

· common*NSPort=<the port for your nameserv specified in step 1)>

· drm*objectname=<the object name you wish DRM to register as in the naming service.>

· drm*cachehome=<the disk path you want DRM to manage>

· drm*maxCacheSize = the size of disk you want DRM to use, in MB or in GB

· coordinate the maxCacheSize with maxSizeConncurrentPinnedFiles. Both are using MBs or GBs.

· drm*localGSIFTPhostname=<hostname where your gridftp server runs for the DRM disk cache>

· drm*useTRM = false <make sure it is false >

· (optional) assign a different file path to each of
drm*logfile,
drm*outputFile,
drm*eventLogFile.

· drm*maxFileSize puts a limit on maximum file size for a request. If not specified, the default is the less of the cache size or 2G. This size is used if no size is specified in a file request.

· drm*showDebugMessages=true by default. If turned to false, then there will not be output messages after DRM is launched. However, the output will be still be written to the file in drm*outputFile.

· drm*generalService=false by default. All users should have their DN in the grid-map file on the machine where DRM runs, to use the DRM service.

· Setting pin parameters is a combination of:

drm*maxSizeConcurrentPinnedFiles

(total size of pinned files. e.g. 500MB)

drm*maxNumConcurrentPinnedFiles

(how many files can be pinned at a time. e.g. 3)

drm*pinDuration

(how long each file will be pinned, in seconds. e.g. 100)

· setting GSIFTP parameters:
drm*localGSIFTPhostname

drm*FTPBufferSize

drm*NumFTPStreams

drm*FTPBlockSize

· drm*cacheFileNamingMode:

by default, files in cache are assigned random numerical names. If you have specific requirements for filenames in cache, we support the following alternatives in drm*cacheFileNamingMode

* ENDFILENAME: source: gsiftp://somewhere/foo will be saved as “foo” in cache
* SUFFIX_ENDFILENAME : source: gsiftp://somewhere/foo will be saved as “xxxx.foo” in cache. Where xxxx is the numeric filename that DRM generates by default.

Note: This option needs to be coupled with the “-retain” option if you are using drm-*.linux to get/put files through DRM.

· To turn off (by simply not specifying ):

drm*appendExistingEventLogs ,

drm*prefetchingLimit,

drm*enableFileSharing,

drm*cacheFileNamingMode.

· Since you are not running TRM, so common*TRM.. entries can be omitted

· To use your own gridmap file, the set it as the value of drm*gridmapfile. Otherwise, the default location at “/etc/grid-security/grid-mapfile” will be assumed

3) start DRM on the machine:

· drmServer.linux drm.rc

If all your entries in drm.rc are ok, you will see “DRM is ready” printed by drmServer.linux.

If you are behind a firewall, you shall add “-OAport xxxx” to the command, where xxxx is the port that is open.

4) open another window, initiate a valid grid proxy. We will test drm-* command now. Your DN needs to be in the grid-mapfile where DRM is running. You need to supply a resource configuration file to these commands. You can create a new file or adding entries to the drm.rc mentioned above.
The commands drm-* will look for the configuration file “drm.rc” in the running directory automatically unless you specify your configuration file via “-conf” option.
In the following example commands, if not specified, we are using “drm.rc” in our environment.

· drm-ping: in the drm.rc file, specify these entries:

common*NSHost=<your corba naming service host>

common*NSPort=<your corba naming service port>

client*DRMName= <the same value as drm*objectname>

run “drm-ping.linux –conf <your drm.rc file name>”

output should be something like this:

Client CONFIGURATION

DRM is alive.

Thu Apr 29 18:45:27 2004

· drm-ls:

sample command (listing the content of directory /tmp from the DRM host machine):

drm-ls.linux –s file:/tmp

The following will generate the same result as the command above,given that DRM was registered as DRMServer on dm.lbl.gov:6171. Other syntax will fail.

drm-ls.linux -s "srm://dm.lbl.gov:6171/tmp"

if you want to see all files in DRM cache, supply an empty string to –s. i.e. drm-ls.linux –s “”. Note that the return will be a list of logical file names.

if you want to view the content recursively, use the ”-r” option

drm-ls.linux –s file:/tmp –r

· drm-status:

supply request id through the –r option. User id will be obtained automatically through the proxy.

sample command:

drm-status.linux -r junmin@dmx.lbl.gov-40a3001e-0

· drm-get:

brings a file from remote (e.g.GSIFTP site or MSS) to local disk
(where client is)

sample command:

drm-get.linux -s gsiftp://dm.lbl.gov/tmp/anExampleToTest -t file:/tmp/hii -b 131019086

Note:

If using –t, then the caller is responsible for making sure the target path exists in the local disk. drm-get will NOT create directories in the client directory.

drm-get is not typically used to retrieve a file that is in DRM already. If you know the file is in DRM, AND you know the file id, then you can retrieve it using drm-get, supplying the file id with “-g” option.

By default (without using –g), drm-get uses the source url as the file id.

· drm-copy:

it has several capabilities, as outlined below.

To bring a file from remote location to local disk (through gsiftp):

* source(remote url): e.g. gsiftp://dm.lbl.gov/tmp/medium.testfile,

specified by “-s”
* target: (can omit) local disk url: e.g. file:/tmp/ok.testfile

specified by “-t”

If you do not provide the target url, the file is transferred into the DRM cache. The address in the cache will be printed out by drm-copy as an output.
* command: (bring a file from dm.lbl.gov to the default disk location)

drm-copy.linux -s gsiftp://dm.lbl.gov/tmp/testfile -b 131019086

* debug output:

WARNING: No entry "TARGETDIR" in the options....

WARNING: Please provide target directory URL....

WARNING: unless targetURL is in the input file....

SRM-COPY: requestid=junmin@dm-4091b039-25cf50d

SRM-COPY: FIDSET_length=1

SRM-COPY: FID=gsiftp://dm.lbl.gov/tmp/medium.testfile

SRM-COPY: STATUSCODE=14

SRM-COPY: RETURN_URL=file:/home/users/junmin/medium.testfile

SRM-COPY: STATUS_EXP=file has been transferred to the address as in returnUrl.

FILE_TRANSFERRED_TO_CLIENT file has been transferred to the address as in returnUrl.

SRM-COPY: FID=gsiftp://dm.lbl.gov/tmp/medium.testfile

SRM-COPY: STATUSCODE=6

To bring a file from remote SRM to local disk:

Command:

drm-copy.linux -s "srm://dmx02.lbl.gov:4000/nersc/users/junmin/ medium_test_2?msshost=garchive.nersc.gov&obj=HRMServer" –t file:/tmp/test5 -b 131019086 -conf drm.rc

Here drm-copy issues a request to DRM (recognized through drm.rc) to contact remote SRM (located at dmx02.lbl.gov:4000/HRMServer) for a file in its HPSS . Once remote SRM has the file ready, DRM will gsiftp to the user specified location /tmp/test5.

Note: that “” is necessary here after –s to embrace ? and &.

To bring a file from remote SRM to local disk through Web Services:

Need to install DRM-GSW. See below DRM-GSW manual for setups and instructions on how to communicate with other SRM Web services.

To bring a set of files from remote to local disk

Create an input file, each line with entries in the following format:

Source_url<space>fileSizeInBytes<space>local_target

Where fileSizeInBytes and local_target are optional. Also, remember local_target should be of “file:” protocol.

Example of a line in such a file with source url, filesize, and local target all specified:

gsiftp://dm.lbl.gov/tmp/testFile 131019086 file:/tmp/vdt

sample command:

drm-copy.linux –f <your_input_file>

NOTE:

· drm-copy option”-sd” is not supported for VDT usage.

3. If you need to clean up the DRM cache:

Shut down DRM server

Go to the cache directory and remove all files.

Trouble shooting:

1. if your request got “Exceeded max allowed size”, it is possibly because:

a. you didn’t specify file size to the client drm-*, and it assumes 2GB by default and sent to DRM

b. in your DRM setting, the cache is less than 2GB or the entry drm*maxFileSize is less than 2GB.

If the above a. and b. fit your set up, you can either provide the actual size (if it is really less than your max file size) or increase the cache size to be more than 2GB.

2. If your locally compiled binary crashes immediately after starting, check your path for GCC and the GCC version that you compiled DRM and other dependencies.

3. if you see DRM gets an error from GLOBUS, try to do a globus-url-copy from your terminal between your site and the other site, usually it would generate the error globus error. You can use “globus-url-copy –dbg” to see the details. Common mistakes include forgetting to add machine name to /etc/host.allow, or the source file does not exist.

4. if you are behind a firewall, you shall add “-OAport xxxx” to the DRM commands (drm-get, drm-copy etc), where xxxx is the open port.

Chapter 2. DRM-WSG

Introduction:

DRM-WSG is a web services gateway to DRM. It allows the interoperability from other SRM implementations.

Web service clients can send calls to DRM-WSG. DRM-WSG component will then forward the calls to DRM, and gets the result back to the user.

How to configure and run DRM-WSG

Requirements:

1. Java version : j2sdk1.4.2_07.

2. Please make sure you have a valid proxy file or service certificates.

3. Please run DRM before you start DRM-WSG.

4. Edit the $DRM_HOME/conf/drm.rc file according to your setup.

The most important ones are as follows :

· common*NSHost=

· common*NSPort=

· drm*objectname=

· wsg*CorbaPort= <CORBA NS port> e.g. 8090

· wsg*RecachingMode=<either true or false>, default false.

true => OVERWRITE mode is yes.

false => OVERWRITE mode is no.

· wsg*servesHRM = false

5. The file $DRM_HOME/conf/drm.rc is used to launch the web service for DRM. It contains the following parameters. Please edit the parameters according to your setup.

wsg*ProxyFile=<proxy_file_location>

wsg*UserCert=<path to USER CERT location>

wsg*UserKey=<path to USER KEY location>

wsg*HostCert=<path to HOST CERT/Service CERT location>

wsg*HostKey=<path to HOST KEY/Service KEY location>

wsg*GridMapFile=<grid_map_file_location>

wsg*HTTPPort=<web_server_port> e.g. 8081

wsg*HTTPGPort=<web_server_secure_port> e.g. 8444

wsg*ServiceHandle=<service_handle>

e.g. /srm/DRM/srmv1

NOTE :

· By default DRM-WSG uses GSI authentication.

· The order of authentication method used: The proxyFile has the highest priority, and then HostCert/Key and then UserCert/Key.

o If proxyFile is given, then proxyFile is used and hostcert/hostkey and usercert/userkey are ignored.

o If hostcert/hostkey or servicecert/servicekey is given without proxyFile, then usercert/userkey is ignored. (Note that a service certificate should be used for a host.)

o If none is given, then it tries to use the default user proxy.

· If WSG started with Host/Service certificates, then any user in the

grid-mapfile can connect to the service after being authenticated.

· If WSG started with UserCert/UserKey or Proxyfile, then only

the user who have started WSG can connect to the service.

Other users in the grid-mapfile are not allowed to access the service.

How to run DRM-WSG:

0. Edit the drm.rc files according to the instruction above.

1. Please have JAVA_HOME and DRM_HOME environment variables

in the shell.

e.g. setenv $JAVA_HOME /usr/local/j2sdk1.4.2_04

e.g. setenv $DRM_HOME /software/drm-1.2.7
<parent directory where the binaries exists>

2. After that just type $DRM_HOME/wsg/srm-wsg-run.sh from the command prompt.

· This step will launch the DRM-WSG server in the port that you specified in the drm.rc file with the service handle and the invocation ID given in the drm.rc file.

· Once the server is successfully launched, it deploys the required

Web services (including OGSA grid related mandatory services) automatically and creates the instance (InvocationId) that you have given in the drm.rc file.

· The server writes the debug log files in the $DRM_HOME/wsg/logs/server_*.log location.

Trouble shooting

1. If it displays GSSException :

Reason : May be your proxy is not valid

2. If it displays IOException : Chain Null

- Reason :

Server may be already running and the process is not exited properly, or port is not released.

- Possible solution:

kill the previous process and start again.

3. If it displays Corba error :

- Reason :

DRM-WSG did not bound to the correct DRM object.

- Possible solution:

Make sure you have the correct common*NSHost, common*NSPort, drm*objectname in the conf/drm.rc file.

Chapter 3. DRM-GWS

Introduction

DRM-GWS is an internal gateway component for DRM to other SRM web services implementations.

Requirements

1. JDK sdk 1.4.2_04 or higher.

2. DRM-GWS must be registered to the same naming service host where DRM is registered. For example, if DRM is registered with dmx.lbl.gov:3000, then DRM-GWS should also be registered on the same naming service host.

3. Please edit the drm.rc file

gws*WsdlPath=<WSDL service handle you want to talk to>

e.g. /srm/DRM/srmv1

WSDL path from the source url takes the priority.

gws*debug=<true or fale>

gws*log4j=<log4j properties file path>

e.g. logs/log4j_gws.properties

gws*MapFile=<map file location>

map file is used only when GWS contacts the glue type web service,

for axis type web service map file is not used.

How to run DRM-GWS

1) Please start DRM, DRM-WSG on site_1

2) Please set JAVA_HOME and DRM_HOME environment variables

in the shell.

e.g. setenv $JAVA_HOME /usr/local/j2sdk1.4.2_07

e.g. setenv $DRM_HOME <parent directory where the binaries exists>

3) Type $DRM_HOME/wsg/drm-gws-run.sh to run the DRM-GWS server.

This step will start the DRM-GWS server. The logs are written in

The $DRM_HOME/wsg/logs/drm-gws-*.log location.

4) (Optionall) Another set of DRM, DRM-WSG server can be run on another site site_2 to test the web services communications.

Chapter 4. SRM-TESTER

Introduction

SRM-TESTER is a tool that tests SRM compatibility of the SRM interface spec v.1.1 as well as the interoperability.And the report results are published on a web site optionally.

Requirements

1. JDK sdk 1.4.2_04 or higher.

2. TESTER_HOME and JAVA_HOME need to be defined.

3. Some of tests will use the DRM configuration from $DRM_HOME/conf/drm.rc.

4. Valid proxy file.

How to run SRM-TESTER

Different property files in the properties directory is used to direct different test.

Example:

properties/drm.properties (to test SRM server)

properties/dcache.properties (to test dCache server)

properties/drmdcache.properties (tocopy a file from dCache server to SRM cache)

properties/drmdrm.properties (tocopy a file from SRM1 server cache to SRM2 server cache)

properties/drmapi.properties (to test the CORE DRM for srm-ping, srm-ls, srm-get, srm-copy)

This test needs -drmhome <path where srm-client-programs are installed> -conf <path where drm.rc file is located>

By default the reports are published in LBNL test web site. http://sdm.lbl.gov/cgi-bin/srm-tester/tester-report-gen.cgi. And the reports can be viewed in http://sdm.lbl.gov/cgi-bin/srm-tester/show-report.cgi.

To publish the reports in different site there are few additional steps needs to be done.

1) Create a directory called srm-tester in your cgi-bin directory. Example:/www/htdocs/cgi-bin/srm-tester

2) copy the scripts at cgi-scripts to your /cgi-bin/srm-tester/.

3) Please give the execute permission to the scripts
chmod 755 /cgi-bin/srm-tester/*.cgi

4) Please decide a data directory, ie, where you want to store your reports. It should be under htdocs directory. Example: /www/htdocs/srm-tester (NOTE: do not create under cgi-bin directory, it will not work)

5) Please chmod 777 to data directory (directory you have created in step 4)

6) Edit the script show-report.cgi my $srm_tester_data_loc_path=<path to the location where you created your data directory> and my $srm_tester_data_loc_url=<url to your data directory>. (NOTE: $srm_tester_data_loc_path and $srm_tester_data_loc_url might differ)

7) Edit the script tester-report-gen.cgi my $srm_tester_data_loc_path=<path to the location where you created your data directory>

Usage of ./srm-tester.sh

NOTE : Some options can be also given in the command, this options will overwrite the parameters given in the properties/*.properties file.

-servicePath (wsdl service path)

example: srm://hostname:portnumber//wsdlServiceName

-proxyfile <path to proxy file>

default: uses the current user proxy from /tmp/x509...

-publishurl <web site location to publish the report)

default : it publishes to LBNL test site

(http://sdm.lbl.gov/cgi-bin/srm-tester/tester-report-gen.cgi)

-nopublish (not publish the result in the web sites)

default: it publishes the result

-s <path to remote machine to get a file>

example: srm:// or gsiftp://

-t <path in your local disk>

-remotetarget <path to put the file in the SRM cache>

-copytarget <path to copy the file in the SRM cache>

-advisorydelete (to do advisory delete after put)

default: dont do advisory delete

-onlygsiftpget (to only gsiftp get) <true | false>

default: false

default : all (will do all the operations)

ping : only pings the SRM

get : get a file from SRM cache or a remote file repository to local disk

put : put a local file to SRM cache.

copy : copy a file from SRM1 to SRM2

-browse : (Browses SRM cache, available only for -drm option)

This option works with LBNL DRM and requires two additional options

-drmhome <path where drm clients are installed>

-conf <path to hrm.rc file> )

-output <path where you want to store the output file>

default: it stores in /tmp/srm-xxx-userlogin.log

where xxx is the different test types

(Output file contains nothing but the outputs printed in the screen)

Some sample commands

1) ./srm-tester.sh -servicePath <SRM service handle> -s <sourceUrl> -t localTarget> -remotetarget <targeturl_for_put> -copytarget <targeturl_for_copy> -advisorydelete -publishurl <web_location_url>

This command will do the following tests.

a) ping the SRM server

b) get a file from SRM cache to local disk,

c) put a small generated sample file called "srm-tester-put.txt” from local disk to SRM cache. This sample file contains only one line "srm-tester test".

d) copy (copy a file from a SRM-Cache to another SRM-Cache)

e) Finally advisory-delete the file in the SRM cache. By default: Advisory delete is not done, it needs to be specified as an option to do advisory delete.

2) ./srm-tester -drm
Note: Before running this test, please edit the properties/drm.properties file.
This command will do the followint tests:

a) Ping DRM server

b) get a file from DRM cache to local disk

c) put a small generated sample file called "srm-tester-put.txt" from local disk to DRM cache. This sample file contains only one line "srm-tester test".

d) copy a file DRM cache to put it the disk path where DRM is running.

3) ./srm-tester.sh –drm –op ping
Note: Before running this test, please edit the properties/drm.properties file. This command will do the following tests.

a) Ping DRM server

4) ./srm-terster.sh -drm -op put -publishurl <publishUrl>
NOTE: Before running this test, please edit the properties/drm.properties file.
This command will do the following tests.

a) ping DRM server

b) put a small generated sample file called "srm-tester-put.txt from local disk to DRM cache. This sample file contains only one line "srm-tester test".

5) ./srm-tester.sh -drm -op copy -s <sourceUrl> -copytarget <copyTargetUrl> -nopublish
NOTE: Before running this test, please edit the properties/drm.properties file.
The command line options will overwrite the parameters given in drm.properties file. This command will do the following tests.

a) Ping DRM server

b) copy the file from sourceUrl to DRM cache given in copytarget

c) the results will not be published in any web site

6) ./srm-tester.sh –drm –browser –drmhome <path to where your drm clients installed> -conf <path to drm.rc file>
Note: 1. Before running this test, edit properties/drm.properties file
2. drm client programs are needed such as drm-ping, drm-ls, etc.
This command will do the four operations (ping, get, put, copy) it also browses DRM cache.

7) srm-tester.sh –dcache
NOTE: Before running this test, please edit the properties/dcache.properties file. This command will do the following operations

a) ping dCache server

b) get a file from dCache server to local disk

c) put a file from local disk to dCache server's cache

d) copy a file dCache server's cache

e) advisory delete the file in the SRM cache. By default, advisory delete is not done, it needs to be specified as an option to do advisory delete.

8) srm.tester.sh –drmdcache
NOTE: 1) Before running this test, properties/drmdcache.properties file needs to be edited. 2) DRM-GWS component is required to run.
This command will do the following operations.

a) ping DRM server

b) copies a file from DCache server's cache to DRM cache

9) srm.tester.sh -drmapi -drmhome <path where drm client programs are installed> -conf <path to hrm.rc conf file>
NOTE : Before running this test, please edit the properties/ping.properties, properties/ls.properties, properties/copy.properties, properties/get.properties files.
This command will do the following operations.

a) ping DRM server

b) test ls

c) test copy

d) test get

Trouble shooting:

1. Connection expection:
Possible reason(s):

· WSG server is not running.

· hostname or port in the source url is not correct.

2. Exception : The AXIS engine could not find a target service to invoke.
Possible reason(s):

· web services path name is not correct.

Chapter 5. DRM-CONF Generator

Introduction

drmconfgen.sh is a tool that runs in both gui and –nogui mode, and generates

the drm.rc configuration file.

Requirements

· JDK sdk 1.4.2_04 or higher.

· DRM_HOME and JAVA_HOME need to be defined.

How to run drmconfgen.sh

drmconfgen.sh has the following options:

-conf <optional conf file path> to modify existing conf file

-template <template conf> (default ./.drmtemplate and required)

-outputfile (name of the output conf file path) (default is ./drm.rc)

-nogui (command line user interactive) (default runs in gui mode)

-help (show this message)

drmconfgen.sh uses the template .drmtempl.rc to generate the drm.rc conf file.

Example:-

drmconfgen.sh -nogui -outputfile <output_file_name>

Runs in the nogui mode and generates the conf file.

If the options –outputfile is not given, then it generates a file called drm.rc in the current directory as default.

NOTE : In the nogui mode, when a list of entries are given to choose, if <ENTER> is hit without providing a value, then the first entry in the list is selected.

Example : [false, true] is provided a choice, if <ENTER> without choosing a value, then the first entry in the list “false” is selected.

drmconfgen.sh –nogui –conf <path_to_existing_drm.rc> -output <output_file_path>

Allows to modify the existing drm.rc file with additional options from the template file.

If –nogui is not used, then it runs in the gui mode.

Chapter 6. SRM-COPY

Introduction:

srm-copy is a client tool for SRM spec v1.1

This section has also some instructions about how to setup srmcp client to work with LBL SRM server.

Requirements:

· jdk1.4.2_04 or higher.

· DRM_HOME and JAVA_HOME needs to be defined.

· Valid proxy.

How to run srm-copy.

Here is the usage and options of srm-copy for SRM spec v1.1.

Usage:

srm-copy [command line options] -s sourceurl -t targeturl

or srm-copy [command line options] -f <file>

where <file> is the path to the xml file containing

the sourceurl, targerurl information.

for requests with more than one file -f option is used.

or srm-copy [command line options] -status requestId

or srm-copy [command line options] -unpin requestId -s TURL

or srm-copy [command line options] -advisorydelete -s TURL

Options: (Command line options will override the options in conf file)

-conf <path to conf file> (default:srm-wsg.rc)

-s <source url>

-t <target url>

-f <fileName> (input file in xml format)

either -s sourceurl and -t targeturl

or -f <filename> is needed.

-td <targetDir> (required only

for get and (copy+get)combo requests

-wsdlpath <web service path> (optional)

Example: /srm/DRM/srmv1

-serviceurl <full wsdl service url>

Example: srm://host:port/srm/DRM/srmv1

-status <integer> (valid requestId returned by server)

-unpin <integer> (valid requestId returned by server)

-advisorydelete

-proxyfile <path to proxy file> (default:user proxy)

-usercert <path to usercert>

-userkey <path to userkey>

-buffersize <integer> (default:1024)

-parallelism <integer> (default:1)

-numretry <integer> (number of retries before client gives up)

(default:3)

-retrytimeout <integer in seconds> (default:300 seconds)

-debug

-logfile <path to logfile>

-output <path to output report file>

Sample:

Get a file from remote SRM

~~....................~~ srm-copy –s srm://<srmaddress:portname>//wsdlpath?SFN=/filepath

-t file:////tmp/get.test

~~....................~~ srm-copy –s srm://<srmaddress:portname>///filepath

-t file:////tmp/get.test -wsdlpath /srm/DRM/srmv1

Get a file from remote SRM using Logical File name

srm-copy –s srm://<srmaddress:portname>//wsdlpath?LFN=/logicalfilename

-t file:////tmp/get.test

Get multiple files from remote SRM

srm-copy –f <xml input file> -td /tmp/test

please see the sample/v1 directory for sample input files.

Put a file to remote SRM

srm-copy –s file:////tmp/put.test

-t srm://<srmaddress:port>//wsdlpath?SFN=/filepath

Put a file to remote SRM using Logical file name

srm-copy –s file:////tmp/put.test

-t srm://<srmaddress:port>//wsdlpath?LFN=/logicalfilename

Copy a file from SRM1 cache to SRM2 cache

srm-copy

-s srm://<srm1address:port>//wsdlpath?LFN=/logicalfilename

-t srm://<srm2address:port>//wsdlpath?LFN=/logicalfilename

Copy a file from SRM2 cache to SRM1 cache and then to local disk

srm-copy

-s srm://<srm2address:port>//wsdlpath?LFN=/logicalfilename

-t file:////tmp/combo.test

-serviceurl srm://<srm1address:port>//wsdlpath

To find status of a request

srm-copy

-status <requestId>

-serviceurl srm://<srm1address:port>//wsdlpath

To Unpin a file in a request

srm-copy

-s <sourceUrl>

-serviceurl srm://<srm1address:port>//wsdlpath

To do Advisory delete a file in a request

srm-copy -advisorydelete

-s <sourceUrl>

-serviceurl srm://<srm1address:port>//wsdlpath

How to run Fermi SRM Web Services client (srmcp) with LBNL SRM server (version 1)

Download Fermi SRM Web Services client tar file and unzip it.

Note: We do not provide or distribute Fermi SRM client (srmcp).

Steps:

0. Please read ${FERMI_CLIENT}/srm/README

1. cd to ${FERMI_CLIENT}/srm directory.

1. setenv SRM_PATH ${FERMI_CLIENT}/srm directory.

2. Fermi SRM client uses ${HOME}/.srmconfig/config.xml.

3. Please copy $DRM_HOME/wsg/wsg-fnal_templates/config.xml to ${HOME}/.srmconfig/config.xml
Please edit the values in the ${HOME}/.srmconfig/config.xml file.

The important parameters that needs to be edited are :

· webservices path=<the webservices handle and Invocation Id>,

e.g. /srm/DRM/srmv1

· mapfile location=<the map file location>,

e.g. $(FERMI_CLIENT)/srm/conf/srm-wsg-axis.map

· proxyfile location=<your proxy file location>

e.g. /tmp/x509……….

4. Please use srm-wsg-fnal_templates/srm-wsg-axis.map file.

· Copy the srm-wsg-fnal_templates/srm-wsg-axis.map file to the ${FERMI_CLIENT}/srm/conf directory.

· Edit the ${HOME}/.srmconfig/config.xml file to point the correct map file path.

5. Some sample commands :

· srmcp srm://dmx.lbl.gov:8443//nersc/users/vijayaln/medium_test srm://dmx02.lbl.gov:8443/tmp/test.file

This command will get a file from the SRM on dmx.lbl.gov to local disk /tmp/test.file.

· srmcp gsiftp://dmx17.lbl.gov:8443//nersc/users/vijayaln/medium_test srm://dmx17.lbl.gov:8443/tmp/test.file

This command will copy a file from a remote gsiftp server on dmx17.lbl.gov to the local disk /tmp/test.file.

· srmcp srm://cmssrm.fnal.gov:8443///resilient/dteam/TEST.c srm://dmx.lbl.gov:8443//srm/DRM/srmv1?SFN=
This command will copy a file from SRM/dCache at FNAL to DRM cache on dmx.lbl.gov.

Chapter 7. SRM-CLIENT

Introduction:

srm-copy, srm-dir, srm-space, srm-request are client tools for SRM spec v1.2

Requirements:

· jdk1.4.2_04 or higher.

· DRM_HOME and JAVA_HOME needs to be defined.

· Valid proxy.

How to run srm-copy.

Here is the usage and options of srm-copy for SRM spec v1.2.

Usage:

srm-copy [command line options] –s <sourceurl> -t <targeturl>

or srm-copy [command line options] –f <file>

where <file> is the path to the xml file containing the

sourceurl, targeturl information.

For requests containing multiple files –f option is used.

or srm-copy [command line options] –s <sourceurl>

-status <RequestToken>

or srm-copy [command line options] –s <sourceurl>

-releasefile <RequestToken>

Options : (command line options will override the options from conf file)

-conf <path to conf file>

-s <sourceUrl>

-t <targetUrl>

-f <filename> (input file in xml format)

Either –s sourceUrl and –t targetUrl

or –f filename is needed.

-td <targetDir> (required only for get requests)

-serviceurl <full wsdl service url>

(example: srm://host:port//wsdlpath)

-userid <userid used in SRM) (default :hi)

-status <requestToken> (valid requestToken returned by server).

-releasefile <requestToken> (valid requestToken returned by server).

-keepspace <true | false> (default: false)

-reservespace <true | false>

(reserves new space at the beginning of request)

-releasespace , <true | false>

(releases the space upon exit)

-proxyfile <path to proxy file>

-usercert <path to usercert>

-userkey <path to userkey>

-logfile <path to logfile>

-output <path to output report file>

-nooverwrite <true | false> (default: overwrites file)

-type <’p’|’d’|’v’> (default:’v’)

(space type : p-permanent, d-durable, v-volatile)

-lifetime <integer> to set token space life time

(default : 100 seconds)

-size <integer> to set token space size in bytes

-renew (renews proxy automatically)

-buffersize <integer> (default:1024)

-parallelism <integer> (default: 1)

-numretry <integer> (default:3)

-retrytimeout <ingteger> (default:300 seconds)

-debug

-help

Examples

Put a file from local disk to remote SRM cache.

This command reservespace at the beginning and

puts a file in the SRM cache and releasespace upon exit.

srm-copy –reservespace –s file:////tmp/put.test

-t srm://<srmaddress:port>//wsdlpath?SFN=/~/put.test

-releasespace

Get a file from remote SRM cache to local disk.

This command gets a file from SRM cache uses the space

already reserverd.

srm-copy –spacetoken <spacetoken> –t file:////tmp/get.test

-s srm://<srmaddress:port>//wsdlpath?SFN=/~/put.test

Copy a file from SRM1 to SRM2

srm-copy –spacetoken <spacetoken>

-s srm://<srm1address:port>//wsdlpath?SFN=/~/copy.test.1

-t srm://<srm2address:port>//wsdlpath?SFN=/~/copy.test.2-

To get status of request (This example shows a put request)

srm-copy –s file:////tmp/put.test

-t srm://<srmaddress:port>//wsdlpath?SFN=/~/put.test

-status <requestToken>

Release file in a request

srm-copy –v2 –releasefile <requestToken>

-s srm://<srmaddress:port>//wsdlpath?SFN=/~/put.test

Reserve a space

srm-space –reserve –type v –size 500 –lifetime 200

-serviceurl srm://<srmaddress:port>//wsdlpath

Release a space

srm-space –release <spaceToken>

-serviceurl srm://<srmaddress:port>//wsdlpath

Release file in a request

srm-space –releasefile <requestToken>

-s srm://<srmaddress:port>//wsdlpath?SFN=/~/put.test

Browse SRM cache

srm-dir –ls –s srm://<srmaddress:port>//wsdlpath?SFN=/~/

Make dir in SRM cache

srm-dir –mkdir –s srm://<srmaddress:port>//wsdlpath?SFN=/~/path

Remove dir in SRM cache

srm-dir –rmdir –s srm://<srmaddress:port>//wsdlpath?SFN=/~/path

Move dir in SRM cache

srm-dir –mv –s srm://<srmaddress:port>//wsdlpath?SFN=/~/path1

-t srm://<srmaddress:port>//wsdlpath?SFN=/~/path2

Remove File in SRM cache

srm-dir –rmfile –s srm://<srmaddress:port>//wsdlpath?SFN=/~/path

Abort Request

srm-request–serviceurl srm://<srmaddress:port>//wsdlpath

-abortrequest <requestToken>

Abort Files

srm-request–s srm://<srmaddress:port>//wsdlpath?SFN=/~/test

-abortfiles <requestToken>

ExtendLifeTime

srm-request–s srm://<srmaddress:port>//wsdlpath?SFN=/UserDN/test

-extlifetime <RequestToken> -lifetime <lifetime>

UserDescritpion

srm-request–s srm://<srmaddress:port>//wsdlpathf

-rid -userdesc

RequestSummary

srm-request–s srm://<srmaddress:port>//wsdlpath

-requestSummary <requestToken>

Some commands to talk to HPSS using SRM

Put a file in HPSS

srm-copy –s file:////tmp/put.test

-t srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Get a file from HPSS

srm-copy –t file:////tmp/put.test

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Copy a file in HPSS

srm-copy

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path1

-t srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path2

Browse a file/directory

srm-directory -ls

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Make directory

srm-directory -mkdir

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Remove directory

srm-directory –rmdir

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Remove file

srm-directory –rmfile

-s srm://<srmaddress:port>//wsdlpath?SFN=mss://garchive.nersc.gov/path

Usage of srm-dir

srm-dir [command line options] -s <sourceurl> -ls

or srm-dir [command line options] -s <sourceurl> -mkdir

or srm-dir [command line options] -s <sourceurl> -rmdir

or srm-dir [command line options] -s <sourceurl> -mv

or srm-dir [command line options] -s <sourceurl> -rmfile

or srm-dir [command line options] -f <file>

where <file> is the path to the xml file containing

the sourceurl information.

for requests with more than one file -f option is used.

command line options will override the options from conf file

-conf <path to conf file> (default:./srm.rc)

-s <dir url>

-f <fileName> (input file in xml format)

either -s sourceurl and -t targeturl

or -f <filename> is needed.

-serviceurl <full wsdl service url>

example srm://host:port//wsdlpath

(required for requests when surl or

or turl did not have wsdl information)

-ls (browse srm cache)

-mkdir (creates dir in srm cache)

-rmdir (removes dir from srm cache)

-recursive <true|false> (recursively removes dir from srm cache)

(default:false)

(to do recursive only for selected dir,

please mention in the input xml file.)

-mv (moves a dir or file in srm cache)

-rmfile (removes files from srm cache)

-proxyfile <path to proxyfile>

(default:from config file)

-usercert <path to usercert>

(default:from config file)

-userkey <path to userkey>

(default:from config file)

-logfile <path to logfile>

(where all debug info are dumped)

(default:/tmp/srm-copy-<loginname>.log)

-renew (renews proxy automatically)

(true | false) (default:false)

(if true is given, prompts for password)

-debug (true | false) default false

-quiet (true | false) default false

suppress output in the console.

this option requires -logfile <path to logfile>

-help show this message.

Usage of srm-request

srm-request [command line options] -s <sourceurl> -abortfiles <RequestToken>

or srm-request [command line options]

-s <sourceurl> -extlifetime <RequestToken> -lifetime <lifetime>

or srm-request [command line options]

-abortrequest <RequestToken>

or srm-request [command line options]

-requestsummary <RequestToken>

or srm-request [command line options] -rid -userdesc <userdesc>

or srm-request [command line options] -f <file>

where <file> is the path to the xml file containing

the sourceurl information.

for requests with more than one file -f option is used.

command line options will override the options from conf file

-conf <path to conf file> (default:./srm.rc)

-s <sourceUrl>

-f <fileName> (input file in xml format)

either -s sourceurl and -t targeturl

or -f <filename> is needed.

-serviceurl <full wsdl service url>

example srm://host:port//wsdlpath

(required for requests when surl or

or turl did not have wsdl information)

-abortrequest <RequestToken> (abort the request)

-extlifetime <RequestToken> (extends the file lifetime)

-lifetime <life time in long>

-requestsummary <RequestToken> (summary of the request)

-rid (get rid)

-userdesc <user description> (default:null)

-abortfiles <RequestToken>

(abort the files transfers in this request)

-proxyfile <path to proxyfile>

(default:from config file)

-usercert <path to usercert>

(default:from config file)

-userkey <path to userkey>

(default:from config file)

-logfile <path to logfile>

(where all debug info are dumped)

(default:/tmp/srm-copy-<loginname>.log)

-renew (renews proxy automatically)

(true | false) (default:false)

(if true is given, prompts for password)

-debug (true | false) default false

-quiet (true | false) default false

suppress output in the console.

this option requires -logfile <path to logfile>

-help show this message.

Usage of srm-space

srm-space [command line options] -reserve

srm-space [command line options] -release <SpaceToken>

or srm-space [command line options]

-s <sourceUrl> -releasefile <RequestToken>

or srm-space [command line options]

-f <inputFile> -releasefile <RequestToken>

command line options will override the options from conf file

-conf <path to conf file> (default:./srm.rc)

-s <sourceurl>

-f <inputFile>

-serviceurl <full wsdl service url>

example srm://host:port//wsdlpath

(required for requests when surl or

or turl did not have wsdl information)

-releasefile <space token > (valid space token

returned by server.)

-keepspace <true | false> (default:false)

(if true, it keep the space, else it releases

the space as well.)

-reserve

-userid (userid used in SRM) (default:user login)

-type <'p'|'d'|'v'> (default : v)

(p - Permmanent, d - Durable, v - Volatile)

-lifetime <integer> to set token space life time

-size <integer> to set token space size in bytes

-release <space token > (valid space token

returned by server.)

(only releases the space associated with token)

-proxyfile <path to proxyfile>

(default:from config file)

-usercert <path to usercert>

(default:from config file)

-userkey <path to userkey>

(default:from config file)

-renew (renews proxy automatically)

(true | false) (default:false)

(if true is given, prompts for password)

-logfile <path to logfile>

(where all debug info are dumped)

(default:/tmp/srm-copy-<loginname>.log)

-debug (true | false) default false

-quiet (true | false) default false

suppress output in the console.

this option requires -logfile <path to logfile>

-help show this message.