root/trunk/doc/install.xml

<chapter>
  <title>Installation</title>
  
  <para>
    This section describes how to install Cobalt. Once these
    steps are completed, Cobalt will be completely functional on the
    system. 
  </para>

  <section>
    <title>Prerequisites</title>

    <para>
      Three prerequisites are required for Cobalt. Each of these,
      their functions and a download location are described below.
    </para>

    <variablelist>
      <varlistentry>
        <term>Python</term>
        <listitem>
          <para>
            Cobalt is written in python. It requires version 2.3 or
            greater. 
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>DB2-python</term>
        <listitem>
          <para>
            This is a library for connecting to DB2 databases from
            python. This is only required for Cobalt on BG/L
            systems. It is available at
            ftp://ftp.mcs.anl.gov/pub/cobalt.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>PyOpenSSL</term>
        <listitem>
          <para>
            PyOpenSSL provides python bindings for OpenSSL. It is
            required in order to support HTTPS on the server side. It
            is only needed on hosts where components execute. 
          </para>
        </listitem>
      </varlistentry>
    </variablelist>     
  </section>

  <section>
    <title>Software Installation</title>
    
    <para>
      Install python, db2-python, and pyopenssl on the server side. On
      SLES9, this can be accomplished by running:
    </para>
    
    <programlisting>
# rpm -ihv \
ftp://ftp.mcs.anl.gov/pub/cobalt/rpms/sles9-ppc64/PyOpenSSL-0.6-1.ppc64.rpm
# rpm -ihv \
ftp://ftp.mcs.anl.gov/pub/cobalt/rpms/sles9-ppc64/cobalt-0.97-1.ppc64.rpm
    </programlisting>

    <para>
      On both the client and server sides:
    </para>

    <programlisting>
# rpm -ihv \
ftp://ftp.mcs.anl.gov/pub/cobalt/rpms/sles9-ppc64/cobalt-clients-0.97-1.ppc64.rpm
    </programlisting>

  </section>

  <section>
    <title>Configuring the Cobalt Component Infrastructure</title>

    <para>
      Cobalt uses https for data security between components and their
      clients. Each machine where components run must have their own
      ssl key. This can be generated by running:
    </para>
    
    <programlisting>
# openssl req -x509 -nodes -days 1000 -newkey rsa:1024 \
-out /etc/cobalt.key -keyout /etc/cobalt.kek
    </programlisting>

    <para>
      Components can be located using static records in
      <filename>/etc/cobalt.conf</filename>, or by using a dynamic
      service location service. The service location component
      (slp.py) is bootstrapped similarly to dns; if a direct reference
      to a component isn't included in
      <filename>/etc/cobalt.conf</filename>, then it is looked up in
      the component listed as "service-location". 
    </para>

    <para>
      Copy the sample cobalt.conf file into place, and change the
      hostname in the service location component line to the one where
      cobalt components will run. Choose a secret password, and place
      this in the password field of the communication section. Once
      all of this is done, the cobalt component infrastructure is
      completely configured. 
    </para>
  </section>

  <section>
    <title>Cobalt Component Startup</title>

    <para>
      Cobalt includes four components for resource management. Each of
      these components provides a specific type of functionality.
    </para>

    <variablelist>
      <varlistentry>
        <term>Service Location Protocol</term>
        <listitem>
          <para>
            The service location component tracks the locations of
            active systems in the component. It uses a heartbeat
            mechanism to detect component failure or exit. It can be
            queried with the slpstat command.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>Process Manager</term>
        <listitem>
          <para>
            The process manager starts, manages, signals, and cleans
            up parallel processes. On BG/L, its functionality is
            implemented using the builtin process management system
            implemented by IBM. The program is
            <filename>/usr/sbin/bgpm.py</filename>. Bgpm requires
            several configuration parameters to be set in
            <filename>/etc/cobalt.conf</filename>. These parameters
            control environment setup for jobs executed. Incorrect
            parameters can cause process execution to fail on nodes.
            This process is started by the cobalt init.d script.
          </para>
          <para>
            Configuration file options are documented in the bgpm(8)
            man page. 
          </para>
          <para>
            On clusters, the process manager uses MPD to start
            processes.  The component is called
            <filename>/usr/sbin/mpdpm.py</filename> and is started
            by the sss-pm init script. Mpdpm doesn't currently take
            any configuration file parameters.
          </para>

          <para>
            On Blue Gene/L, the mpirun command must work for users
            on the host running bgpm.py. This is usually the service
            node. In most cases, rsh/ssh must be reconfigured to allow
            users to ssh from the service node to the service
            node. (This allows the mpirun frontend to properly contact
            the mpirun backend)
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>Queue Manager</term>
        <listitem>
          <para>
            The queue manager handles all aspects of action
            aggregation related to jobs. For example, it uses the
            process manager interfaces to run user jobs, as well as
            prologue and epilogue scripts. It also handles job stdio
            handling on systems without a global shared filesystem.
          </para>
          <para>
            Cqm is the cobalt implementation of the queue manager. It
            is common to both BG/L and clusters, though it must be
            configured slightly differently for each. It uses a number
            of parameters in the <filename>/etc/cobalt.conf</filename>
            that control the behavior of jobs and which external
            systems are used. The queue manager currently has support
            for file staging (for machines without global shared
            filesystems), and basic support for allocation
            management. This daemon is started by the cobalt init.d
            script.
          </para>
          <para>
            All configuration file options are documented in the
            cqm(8) man page.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>Scheduler</term>
        <listitem>
          <para>
            The scheduler controls resource allocation for job
            execution. It tells the queue manager when and where to
            run jobs. Due to differences in scheduling requirements,
            Blue Gene/L systems and clusters require different
            schedulers. 
          </para>
          <para>
            Bgsched is the scheduler for Blue Gene/L systems. It
            internally tracks partition state and performs DB/2
            queries to ensure coherent partition usage in case of
            problems. Bgsched currently only accepts configuration
            options to control database connection parameters. These
            options are documented in the bgsched(8) man page. It is
            started by the cobalt init.d script.
          </para>
          <para>
            Describe the cluster scheduler here.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>Allocation Manager</term>
        <listitem>
          <para>
            The allocation manager tracks users, their project
            memberships, and time allocations. It is used by the
            scheduler to control resource allocation. A common
            allocation manager is used on cluster systems and Blue
            Gene/L systems. It currently has no configuration file
            options, and isn't started up by the cobalt init.d script
            yet. 
          </para>
        </listitem>
      </varlistentry>
    </variablelist>

    <para>
      Once each of these components is started, an entry will appear
      in the service location component. This can be displayed with another
      call to <filename>/usr/sbin/slpstat.py</filename>. 
    </para>

    <para>
      Each component can also be queried with a component specific
      tool. For example, the queue manager can be queried with the
      <filename>cqstat</filename> command. See the clients directory
      for other commands that can connect to cobalt components. 
    </para>
  </section>

  <section>
    <title>Basic Component Testing</title>
    <para>
      Need to rewrite.
    </para>
  </section>
</chapter>