#F>========================================================================= # localdev.INSTALL # RCS: $Revision: 1.11 $ $Date: 2001/08/28 16:39:17 $ $Author: prentice $ #========================================DISCLAIMER========================= # This file has been developed by and/or for the U.S. government within the # Forecast Systems Laboratory by government and/or contract personnel. # As required by government regulations, this file is in the public # domain, and is furnished "as is", without technical support, and with # no warranty, express or implied, as to its usefulness for any purpose. #F>>=====================================description (optional)============= # This file describes how to install localdev package. It is valid for # versions 2.0 and higher of localdev. #F>>>======================================================================= LOCALDEV INSTALLATION SUMMARY ============================= The 'localdev' installation procedure is designed to provide flexibility with regard to which platforms are supported. The localdev package itself is written in Perl, and thus need not be built for individual platforms. Localdev operation does depend, however, on several aggregate GNU packages. These required GNU packages are herein referred to as 'aggregates'. The localdev package contains special 'relocatable' versions of these aggregates. Although each of them does need to be built for each individual platform that localdev is to run on, the process has been automated. Each aggregate package is included within the localdev package in its entirety, including source code and documentation. The process of building aggregates as part of the localdev installation process is described herein. Generally, the localdev installation procedure consists of the following steps, each of which is explained in detail later in this file. 1) Choose a location and ownership for localdev Plan where you want localdev installed, and under which account and group. 2) Extract and run the installation script Expand a localdev archive into a selected installation directory (DISTDIR). A 'localdev-###' directory is created within DISTDIR to hold all installed localdev files (where ### is the current localdev version). If installing into a directory already containing a version of localdev, then all custom configuration files are preserved, and can be updated in a subsequent step. All other (non-custom) files are replaced. 3) Update all custom configuration files. Run a script to copy custom files that you created in a prior version of localdev. A few manual files may need to be manually merged. The script will identify these for you. 4) Configure GNU aggregates For each platform that you want localdev commands to work on, you need to configure a set of GNU executables that can be run by that platform. GNU aggregates are a set of GNU packages aggregated with the localdev distribution. 5) Install localdev accessories, if desired. Accessories provide optional capabilities that are appropriate for specialized needs. The nwscm accessory is the only one currently in existence. 6) Make localdev accessible Take a few steps to make localdev accessible to developers and to validate the installation. PREREQUISITES ============================= At present, the localdev package is distributed only as a compressed tar archive. You therefore need to have a version of the following utilities available (built, installed, and shell-accessible) in order to expand the archive. * gunzip (available from GNU) * tar (available from GNU, and elsewhere) * csh or tcsh (for installation scripts) Perl 5.003 (or higher) must be available (built, installed, and shell-accessible) on each machine from which developers will run localdev scripts. Furthermore, the perl executable must be accessible via the path /usr/local/bin/perl on each machine. Perl need not be available to install localdev. Installation scripts are written in cshell, and have been tested under the tcsh shell. In order to install localdev, you will need to be able to build GNU software aggregate packages on each platform upon which developers will run localdev scripts. Once built, localdev scripts automatically access the scripts that are approriate for the current platform. Building GNU packages requires the capability to build software written in the C language. The following software must be available (built, installed, and shell-accessible) only on machines where you'll be building localdev aggregates. Once these are built, localdev operation does not depend on access to these development tools. * A C compiler, preferably GNU's gcc compiler * Any utilities required by the compiler (e.g., binutils for gcc). * The standard collection of header files and libraries normally used when developing software with a C compiler. Localdev relies on NSF to make file hierarchies containing production areas available to developers on their workstations across a local area network. Localdev must be installed in a location that is accessible to anyone that is to use it. While this can be done by copying an installed localdev to multiple machines, it is usually more convenient to simply mount the installation directory on multiple machines to provide accessibility. This makes it simple for all users to share a common set of configuration files. If multiple installations are supported, then any custom configuration files that are created would have to be manually copied to each installed version of localdev. LIMITATIONS ============================= At present, the localdev package has been tested only on UNIX systems, which all share common notions of file formats, filesystem conventions, and basic utilities. Although both GNU software and Perl are intended to be portable to a wide variety of platforms, it is uncertain that this package will perform flawlessly (or at all) on any particular platform other than those upon which it has been tested (HP-UX and Linux). These scripts will not run, for example, on any system that does not support symbolic links. The localdev package assumes use of the RCS code management system. There is, further, no mechanism currently provided for importing revisions maintained within SCCS files into localdev production areas. Hierarchies of files outside a revision control system, or those within RCS or CVS revision control systems, can easily be imported to localdev in their entirety. INSTALLING LOCALDEV ================================================= 1) Choose a location and ownership for localdev ------------------------------------------------- The installed localdev package is relocatable. You can always tar it up and expand it somewhere else, giving the account performing the relocation ownership. It is more efficient, however, to make a good choice of location and ownership at the outset. The account performing the installation will own the installed localdev files. This need not be the root account, and need not be an account that owns project files or production areas, though this may be convenient. The (account and group) ownership of the localdev installation directory controls who may: * use localdev scripts (read-only access required) * create, modify, and delete localdev configuration files (read-write access required) The installed files are readable to everyone, so the real decision to make about ownership is what account has ownership, and hence write access. Individual configuration files can be made group-writable as desired to expand write access to more individuals. Most developers need only read access to the localdev installation directory. Any personal or production area configuration files that they create through localdev are stored elsewhere. Those who are to be responsible for creating and maintaining configuration template files, however, must be given write access to the localdev installation directory. The location of installed localdev files is relevant to localdev operation. The installation directory (which will be called DISTDIR herein) will need to be accessible across the network to all machines from which developers will be invoking localdev scripts. This location must be accessible VIA THE SAME PATH on all machines. This location need have no relationship to locations where localdev production areas reside. The installation procedure will create a localdev-### directory within DISTDIR (where ### is the current localdev version). The usual installation procedure is to create a symbolic link named DISTDIR/localdev that points to DISTDIR/localdev-###. (Temporarily, localdev2 and localdev links both exist, so as to make both localdev 1.x and localdev 2.x available simulataneously.) The default setup that each developer will use to access localdev will use this generic link(s). This will allow you to update localdev without disrupting developer access. They'll automatically switch to the new version when you change this link. GNU aggregate packages will be built in DISTDIR/ldev_aggs, and then installed into localdev-###. As each aggregate is installed, the aggregate build directories within DISTDIR/ldev_aggs are deleted, so this disk space is needed only temporarily. If you are short on disk space, you can create DISTDIR/ldev_aggs as a link or mount point to another disk partition prior to installation. This will significantly reduce space requirements within the disk partition containing DISTDIR and DISTDIR/localdev-###. Disk requirements for installation and operation are summarized below. The disk space required for operation is dependent on the number of platforms that you want to support. Aggregate executables must be built for each platform that you want to be part of your network-wide development environment. Space for localdev operation (DISTDIR/localdev-###) ---------------------------------------------------------------------------- localdev archive (minus aggregate source) 3.5 megabytes installed aggregates 12-42 megabytes per platform (quite a range -- HP-UX is the high and Linux is the low) temporary storage for aggregate source 6.5 megabytes ----------------------------------------------- total 10 + up to 42 megabytes per platform Extra space for installation (DISTDIR/ldev_aggs) ---------------------------------------------------------------------------- build scratch space 50 megabytes temporary storage for built aggregates 12-42 megabytes per platform ----------------------------------------------- total 50 + up to 42 megabytes per platform The required build scratch space assumes that the expanded source for each aggregate is created just before being used and deleted just after being used, so that only one expanded aggregate exists at a time. If you want to keep source for each aggregate in expanded form, then you may need up to 12 additional megabytes of disk space. 2) Extract and run the installation script ------------------------------------------------- a) Login to a machine having access to the DISTDIR using the account that you have selected for installation. b) Make the DISTDIR directory that you have selected the current directory. If DISTDIR=/usr/local, for example, then the installation script would install version 2.0 of localdev into /usr/local/localdev-2.0. c) Copy the localdev distribution (such as localdev-2.0.tar.gz) to a location that can be accessed from this location. d) You will need access to tar and gzip utilities in order to expand the localdev archive. The localdev distribution contains scripts that perform most of the installation. All that you need to do manually is to extract the main installation script from the archive file. Execute the commands below, substituting values for the following variables: DISTDIR = the path to your installation directory CMDPATH = the path to tar and/or zcat utilities ARCHIVEPATH = the path to the localdev distribution archive ### = localdev version number (from the distribution archive: localdev-###.tar.gz) (check for zcat and tar accessibility) which zcat (should yield a pathname -- CMDPATH) which tar (should yield a pathname -- CMDPATH) (place zcat and tar in your path, if required) csh: set path = ( CMDPATH $path ) ksh/sh: PATH= CMDPATH:$PATH; export PATH (extract the installation script) cd DISTDIR zcat ARCHIVEPATH/localdev-###.tar.gz | tar xf - ldev_install e) Run the ldev_install script that should now reside in DISTDIR. It will prompt for information as required. When the script completes, a localdev-### directory will exist, containing all installed files contained in the archive. You should now see the following directories within localdev-###. bin contains localdev Perl scripts config contains localdev configuration files doc contains localdev documentation in HTML format aggsrc contains (buildable) aggregate source distributions lib contains localdev library files platforms contains pre-built aggregates etc contains scripts used during installation If it looks okay, then remove the installation script: rm -f ldev_install 3) Update all custom configuration files. ------------------------------------------------- If you are installing localdev for the first time, then you can skip this step. The purpose of this step is to compare the current localdev installation with a prior one, and to copy customized files from the prior installation to the current one. Scripts are provided to help with this procedure, and to indicate when file incompatibilties between versions require recreating a custom file from scratch. a) From DISTDIR/localdev-###, run etc/ldev_customize PRIORPATH where PRIORPATH is the absolute path of a prior localdev-### directory that you want to take customization files from to update the current localdev installation. This will generate a report of what needs to be done in order to recreate your customizations within the current installation. b) If you want to proceed with the update, then from DISTDIR/localdev-### run etc/ldev_customize -c PRIORPATH This will perform the simple copy operations required to recreate your customizations within the current installation. It will also indicate which types of operations remain to be performed manually in order to complete your customization. c) Perform any indicated manual merges of old (customized) files with newly installed files that seem desirable. You have the option of not carrying forward any of these customizations. 4) Configure GNU aggregates ------------------------------------------------- Localdev installation scripts give you several ways of making GNU aggregates accessible to all platforms that you'll be accessing localdev scripts from. First, realize that platform is governed by three attributes: osname: the operating system (/bin/uname) osver: the operating system version (/bin/uname -r) arch: the machine architecture (often /bin/arch) Note that on HP-UX systems, architecture must be derived from a combination of your model number (/bin/model) and a search for a match in the file /usr/sam/lib/mo/sched.models. You need not compute your platform manually. The config_aggs script, when run with the -b option, will compute the platform and then allow you to either quit or select one of several ways to configure GNU aggregates for the current platform. You will run config_aggs with the -b option once from each platform of interest, accumulating executables or links within a DISTDIR/ldev_aggs/platforms directory. Each of these runs configures the current platform for use with localdev. Running config_aggs without options will show you which platforms you have configured. When all desired platforms have been configured, then you can run config_aggs with the -i option to install them all into localdev. Your installed localdev remains unaffected until this time. You can then eliminate the ldev_aggs directory. At a later time, you can add another platform by running 'config_aggs -b' again, followed by 'config_aggs -i'. By default, the new platform will be added to what has already been configured. If you need to remove a platform, then you can either start from scratch (telling 'config_aggs -i' to delete all prior installed platforms), or you can manually delete appropriate directories under DISTDIR/localdev-###/platforms. Given the above overview, the following presentation of the various build actions supported by config_aggs will make more sense. The config_aggs script provides the following alternatives for each platform: internal: build a set of aggregate executables for this platform within localdev external: link localdev with a set of compatible external aggregate executables emulate: cause the current platform to use a set of aggregate executables configured for a platform having the same operating system and a compatible architecture You always have a choice of configuring a default set of aggregates for any version of the current operating system, or configuring a set of aggregates specifically for the current version of the current operating system. If you change you mind about how you want to configure a platform before running 'config_aggs -i', then just rerun 'config_aggs -b'. It offers you the option of deleting the platform configuration, or will replace it if you choose a different option. The most straightforward configuration is to build one set of internal aggregates for each combination of operating system and architecture that you run. You usually need not configure platforms for specific versions of each operating system. You can use emulation for some architectures, but doing so can slow performance. On the other hand, building more aggregates will consume more disk space. You can use external aggregates if you have them or want to create them. Choose the combination that works best for your site. Note that aggregates built and installed within localdev have been altered to be relocatable, so long as they are all installed into a single directory that is prominently placed on the command path. This allows the installed localdev package (including its aggregates) to be relocatable. External aggregates will expect to find each other in specific locations that were determined at the time of build. You can use externally built aggregates on a particular platform only if all required aggregate packages are available on all machines that run that platform, and are accessible by a single set of pathnames. Perform the aggregate configuration process as described below. For each platform that you want to configure for localdev: a) Login to that platform using the same account that you installed with. b) Check that gcc or another C compiler is in your command path. If gcc is not in your path, then etc/config_aggs will prompt you for the name or path of a compiler. c) Perform the following commands to configure aggregates for this platform: cd DISTDIR/localdev-### etc/config_aggs -b Aggregate packages are expanded in DISTDIR/ldev_aggs, and installed in various platform-specific directories under DISTDIR/ldev_aggs/platforms. NOTE: If you want to preserve expanded GNU source distributions within the DISTDIR/ldev_aggs directory, then manually expand them prior to running config_aggs. This will consume additional disk space, but will cause config_aggs to run a 'make distclean' at the conclusion of each build, rather than removing the expanded distribution. To expand an aggregate manually, perform the following commands (where GNUDIST is the name of the distribution) mkdir DISTDIR/ldev_aggs cd DISTDIR/ldev_aggs zcat ../localdev-###/aggsrc/GNUDIST.tar.gz | tar xf - (repeat for each GNUDIST.tar.gz file in aggsrc) d) At any time, you can run 'etc/config_aggs' (without arguments) to list which platforms you have successfully configured. e) If a build does not succeed, then perform it manually, using the instructions in the top of config_aggs. You will need to expand the GNU aggregates manually, and then step through the commands executed by config_aggs. Keep track of both build procedure changes and source file changes. When you get it to work, create a modified version of both config_aggs and the aggregate archive. Make sure that your version of config_aggs uses your modified archive. You can reuse these files each time that you update localdev. If you send these files to localdev's auther, they might even find their way into the next localdev release. If the supplied files do not build successfully, then you will want to derive a customized version of config_aggs that does successfully build on all of your platforms, using any special GNU aggregate archive files that you have created. This will allow you to easily rebuild aggregates next time you need to upgrade localdev. With all aggregates built, run: etc/config_aggs -i This will install all built aggregates into localdev. After running this command, you can remove DISTDIR/ldev_aggs, if desired. 5) Install accessories (optional) ------------------------------------------------- It is often convenient to install localdev accessories, such as nwscm, at this point. Then the process of making localdev accessible, below, automatically incorporates the accessories as well as localdev proper. Refer to accessory installation instructions to determine how to proceed. 6) Make localdev accessible ------------------------------------------------- a) Login to a primary system to an account that will be using localdev. b) Run the following commands to validate accessibility to Perl and localdev. cd DISTDIR/localdev-### etc/ldev_verify The script will validate that: 1) Perl version 5.003 or higher is installed and accessible via /usr/local/bin/perl. 2) Sourcing etc/csh_setup_trial places the installed version of localdev within your command path, making its commands accessible. 3) Libraries and other files required by localdev are accessible and not corrupted. c) If any problems are found by ldev_verify, then correct them. Rerun ldev_verify to ensure that all problems have been eliminated. d) If this is the first time you've installed localdev on a particular set of machines, then you should verify access by repeating steps (a) through (c) on each machine that will access localdev. One way to spread the responsibility for this test among developers is to send out a request as part of (f), below, requesting developers to run ldev_verify and report any problems found. Remedy any such problems. Rerun ldev_verify on appropriate machines to ensure that all peoblems have been eliminated. e) install documentation There is a localdev web site at www-md.fsl.noaa.gov/adr/localdev. This site contains a complete set of current localdev documentation. Developers at your site may be able to access localdev documentation there. If you want local access to localdev documentation, however, the 'DISTDIR/localdev-###/doc' directory contains all localdev documentation in HTML format. You can use them where they are, or move them somewhere else, as desired. f) Publish a notice to all users and production area owners indicating that an upgrade will be taking place. There are two sample messages provided with localdev that you can customize and use as desired. The file etc/new_notice is appropriate for a group that is using localdev for the first time. They don't have existing production areas to worry about. The file etc/update_notice is appropriate for groups that are already using localdev, and are receiving an update. * The notices indicate a time at which you will change over to the new version of localdev. * The notices indicate to all users the location of the file DISTDIR/etc/csh_setup that should be used to set up access to localdev from their account. * The notices indicate that the generic localdev link will be changed to point to the new version of localdev. * The notices indicate where localdev documentation can be found. * The update notice will indicate that all production areas will go off-line just prior to the change over, and that production area owners must run dv-repair on all baselines prior to bringing them back on-line. g) Perform the change over to the new version, performing all of the steps outlined in the relevant notice. Verify that you can access the new localdev package via etc/csh_setup by doing the following: source DISTDIR/etc/csh_setup dv-help -V You should receive an indication of the localdev version number that matches the version that you just installed. If this does not work, then check your generic DISTDIR/localdev (or DISTDIR/localdev2) link. It may be pointing to the wrong place. h) If this is a new installation, contact those responsible for setting up production areas within localdev hubs. Work with them to decide on the locations of project hubs, and ensure that sufficient disk space is available for project needs. Assist them in deciding which accounts will own each hub, and the production areas that each will initially contain. At least one hub and production area of interest to a developer must exist before he or she can use localdev. Actual creation of production areas and the baselines within them should be performed by whomever is going to maintain them. The localdev package is now ready for use. All that developers need to do in order to access localdev scripts is source one of the localdev setup files (DISTDIR/localdev2/etc/csh_setup, DISTDIR/localdev2/etc/csh_setup2, DISTDIR/localdev2/etc/ksh_setup, or DISTDIR/localdev2/etc/ksh_setup2) to alter their shell command paths. Then it will be up to those who have been designated production area administrators to use appropriate production area owner accounts to create localdev production areas. Individual developers then place these production areas within scope of their personal localdev configuration, and can then utilize localdev for software development. The key piece of information that all developers will need is the location of the installed localdev package. It is highly recommended that the location be (or appear to be) identical on all development machines. If mounted in the same location on each machine, you can always use symbolic links to create a virtual common location through which all developers can access it. Similarly, directories that are to contain production areas must be made accessible to all relevant development machines. It is undesirable for developers to directly use some RCS and BCS commands to alter localdev source files. Doing so can compromise the integrity of a localdev source area, especially if inconsistent versions of RCS commands are accessed. For this reason, localdev provides wrappers for RCS commands within its bin directory. These access the correct versions of the RCS commands, and provide some safety mechanisms to eliminate accidental use of dangerous commands. If this directory is placed prominently in one's command path (as is done by etc/csh_setup), then other, incompatible versions of these commands will be effectively blocked. It is simple to block commands that you do not wish executed when a localdev environment is active. Add additional links to dv-block in the localdev bin directory for any scripts or commands that you want to block. You can also copy dv_block to another directory along with its links, if desired. 6) Optional steps ================================================= Each of the steps below is optional. You can ignore them or perform them at any time after installation. a) Recover disk space If you want to reduce the size of the installed localdev installation, then you can delete the aggregate archives. You can always recreate them by re-expanding the original localdev archive somewhere, and they need not be kept on-line for localdev operation. NOTE: If you have modified versions of these archives, then be sure to save them somewhere, and leave a build.README file behind in the DISTDIR/localdev-### directory to indicate where they are. cd DISTDIR/localdev-### rm -Rf aggsrc You can also save a small amount of disk space by removing unneeded files from the 'doc' directory (see step 6e, above). Make sure, however, before deleting this documentation, that developers have access to localdev documentation in some form elsewhere. cd DISTDIR/localdev-### rm -Rf doc b) Relocate/Distribute the installation If you have installed localdev where you want it, then this step can be skipped. All developers must be able to access the installation directory from their machine in order to use localdev. If you want to move it, either now or in the future, then simply place it into a tar archive and move it, as follows: cd DISTDIR tar cvf localdev-###.installed.tar localdev-### cd NEWDISTDIR umask 002 tar xvf DISTDIR/localdev-###.installed.tar ln -s DISTDIR/localdev-### localdev (manually edit DISTDIR/localdev/install_home -- see note below) where: DISTDIR = the current localdev installation directory NEWDISTDIR = the new localdev installation directory ### = the localdev version number NOTE: The ownership of a localdev installation is determined by the account used to expand the tar archive. See considerations regarding ownership in step 1, above. Once you have the tar archive, you can install it in as many places as you need to. Note, however, that localdev configuration and registry files are shared only by developers accessing the same copy of the localdev installation directory. IMPORTANT NOTE: There will be a file 'install_home' within the localdev-### directory. It contains the network-wide path to the installed localdev directory. You must manually edit this file each time you relocate localdev, so that it contains the desired path. This file is required because the presence of mount points sometimes confuses a script's ability to find the network-wide path to DISTDIR/localdev.