LIGOtools Administration Guide

for 'ltadmin' package, version 2.3

Peter Shawhan
May 21, 2004

Overview of LIGOtools
Structure of the LIGOtools Directory Tree
Initial Setup of LIGOtools
Working with LIGOtools Packages
Preparing a LIGOtools Package
Maintaining the LIGOtools Web Site

Overview of LIGOtools

The goal of the LIGOtools project is to simplify the installation and use of various LIGO/LSC software tools. A LIGOtools installation is basically a directory tree with a particular organization. This directory tree can be located on any filesystem. We refer to the base of the directory tree as the "LIGOtools directory", and it is strongly recommended that the environment variable LIGOTOOLS be set to contain the full pathname for this directory.

All LIGOtools software and documentation is available from the LIGOtools web site, http://www.ldas-sw.ligo.caltech.edu/ligotools.

Structure of the LIGOtools Directory Tree

The LIGOtools directory normally has eight subdirectories: packages, bin, lib, include, matlab, root, java, and config. The packages directory is used to store software "packages", each of which may consist of a set of executable programs, libraries, Matlab scripts, ROOT macros, etc. The user normally is not concerned with the contents of the packages directory, but works only with the other six directories, which contain symbolic links to the "active" versions of various packages. For example, a utility called "checkit" from the "veritools" package might be located in $LIGOTOOLS/packages/veritools/1.2/bin, with a symbolic link to it in $LIGOTOOLS/bin. Thus, a user only has to add one directory, $LIGOTOOLS/bin, to his/her PATH environment variable, and will have access to all the executables provided with current and future LIGOtools packages. The lib, matlab, root, and java directories serve similar purposes, concentrating elements from various packages into a single location (at least from the point of view of the user). The config directory is used to store certain information needed by some LIGOtools utilities.

Within the packages directory, there is one subdirectory for each package which has been installed. Within each package's directory (e.g. $LIGOTOOLS/packages/veritools), there is one subdirectory for each version of the package which has been installed, plus a symbolic link called active which is a link to the subdirectory of the currently active version. A directory for a specific version (e.g. $LIGOTOOLS/packages/veritools/1.2) can have anything in it, though typically it will include a bin directory and perhaps lib, include, matlab and/or root directories.

A few basic tools are needed for the installation and administration of LIGOtools software packages, such as: perl (anywhere in your PATH), /usr/bin/env, tar, gzip. That should be enough to install "binary" distributions. In some cases, building "source" distributions may require specific compilers, e.g. gcc.

Initial Setup of LIGOtools

To set up a LIGOtools directory, follow the instructions at http://www.ldas-sw.ligo.caltech.edu/ligotools/setup_info.html to download the latest ligotools_init_*.tar file, un-tar it, and run the ligotools_init script. As you follow the script's prompts, you will have to decide whether you want the LIGOtools bin, lib, etc. directories (which will contain symbolic links to the actual files in each package) to reside under the main LIGOtools directory, or somewhere else. For instance, if your group is already accustomed to keeping utility programs in a directory called /usr/local/ligo/bin, then you may wish to use this as the LIGOtools bin directory since it is probably already in everyone's PATH environment variable. Otherwise, you should probably just let the bin directory be in its default location, as a subdirectory of the main LIGOtools directory, and modify your PATH accordingly. (If you do use a non-standard location for the bin directory, then $LIGOTOOLS/bin will be a symbolic link to that directory, so you can always access programs via $LIGOTOOLS/bin.)

At the end of this initial setup, you will be prompted to execute the following command in your shell:

    eval `<dir>/bin/use_ligotools`

where <dir> is the main LIGOtools directory you just set up. This command sets the LIGOTOOLS environment variable for you and ensures that the $LIGOTOOLS/bin directory is in your PATH, etc. You can put this command in your .login or .cshrc file to be executed whenever you log in. This command works with all popular shells (sh, bash, ksh, csh, tcsh) and can safely be executed multiple times.

The initial LIGOtools setup installs the ltadmin package, which provides a few scripts (available in $LIGOTOOLS/bin) for managing the LIGOtools directory structure. You will have to install all the other packages, as described below.

Working with LIGOtools Packages

The main administration task is to install a new package, or a new version of an existing package. There are two ways to do this. The first way is to run the ligotools_update utility, which checks the distribution web site for new packages and/or new versions of existing packages, then (after asking the user for approval) downloads and installs them. This is the recommended method due to its simplicity. The only catch is that ligotools_update is a script which uses the tclshexe program from the tclexe package, so you must manually download and install the tclexe package before you can use the ligotools_update utility.

To manually install the tclexe package (or any other package), go to the LIGOtools web site and follow the directions to download the distribution file into any convenient directory (it does not matter where), then type ligotools_install <file>, where <file> is the name of the file. If your LIGOTOOLS environment variable is set, then the script knows where to install the package; otherwise you will have to type in the location of the main LIGOtools directory by hand. The script determines the package name and version from the filename, creates the necessary directories within the LIGOtools directory tree, and unzips/untars the contents of the distribution file into that directory. If the file was a "source" distribution file, then make is invoked to build executables, libraries, etc. Finally, assuming that you have indicated that this new version is to be the "active" version, the ligotools_install script executes ligotools_activate to create the necessary symbolic links in $LIGOTOOLS/bin, $LIGOTOOLS/lib, etc. to provide access to the newly installed package. Note that you may have to type rehash to make your shell aware of any new executables added to $LIGOTOOLS/bin.

As hinted in the previous paragraph, there is a distinction between installing a package (i.e. adding it to the LIGOtools directory structure and building it if necessary) and activating it (i.e. creating links in $LIGOTOOLS/bin, etc.). This allows you to keep multiple versions of a package and to switch between them if necessary using the ligotools_deactivate and ligotools_activate utilities, though normally this is probably only necessary when testing a development version. There is also a ligotools_uninstall script which completely deletes the specified version of a package, and is safer than simply deleting that version's directory since it warns you and then deactivates that version first, if necessary. Finally, there is a script called ligotools_cleanup which removes any orphaned symbolic links that might have been left over from some incomplete operation, though this should not normally happen.

To find out the arguments required by any of these administration scripts, just type the name of the script without any arguments.

Starting with version 1.5 of the ltadmin package, a log of all administration operations (install/uninstall, activate/deactivate, cleanup) is kept in $LIGOTOOLS/config/admin.log.

Technical note about the symbolic links created by ligotools_activate: this script chooses whether to use relative or absolute pathnames for the symbolic links it creates, depending on whether the target directory (e.g. $LIGOTOOLS/bin) is a real directory or is a symbolic link to some other directory, respectively. Relative pathnames are used in the "usual" case in which everything is kept within real subdirectories of $LIGOTOOLS; this is supposed to permit the disk containing the top-level $LIGOTOOLS directory to be mounted with different names on different machines, but in practice, some LIGOtools packages may rely on hard-coded pathnames set up when the packages were installed.

Preparing a LIGOtools Package

Assemble all needed components in a directory, typically with a few subdirectories such as bin, src, doc. Create a Makefile which does a complete build in response to 'make'. The Makefile should also include the following pseudo-targets:

clean -- Deletes everything produced by the build process, leaving only those files that came with the original distribution.
dist -- Creates a "source distribution" file using tar and gzip, giving it a name of the form <package>_<version>_SOURCE.tar.gz. In general, this file should include the minimal set of files needed to recreate the package elsewhere, including the Makefile.
bindist -- Creates a "binary distribution" file, giving it a name of the form <package>_<version>_<systype>.tar.gz, where <systype> is the name of the operating system as returned by the ltsystype command (which is included in the ltadmin package), e.g. SunOS or Linux. In general, this file should include only those files actually needed to use the package, and should not include the Makefile.

If a package requires no building and will run on all operating systems (e.g. it consists only of scripts), then the dist pseudo-target should instead create a file of the form <package>_<version>_ALL.tar.gz, and the bindist pseudo-target may be omitted.

You are encouraged to model your Makefile after a Makefile from an existing LIGOtools package! Contact Peter Shawhan (shawhan_p@ligo.caltech.edu) if you need help with this.

The LIGOtools administration scripts (ligotools_install, ligotools_activate, ligotools_deactivate) have a default behavior which is appropriate for many packages; for instance, ligotools_activate creates symbolic links for all files in the package's "public" directories (bin, lib, include, matlab, root, and java). However, there are a few ways to explicitly control this behavior:

If there is an executable script called postinstall, then that script is invoked after the package is installed (and after make was invoked, if it was a source distribution). This can be used to create machine-specific startup scripts, etc.
If there is an executable script called activate, then the ligotools_activate utility executes this activate script instead of creating the default symbolic links in $LIGOTOOLS/bin, $LIGOTOOLS/lib, etc. For instance, this can be used to create "public" links for only a few selected programs, libraries, etc. You should make the activate script smart enough to decide whether to use absolute or relative pathnames for the symbolic links, based on whether the bin directory is itself a symbolic link; see $LIGOTOOLS/packages/dataviewer/5.3/activate for an example of how to do this. Similarly, the ligotools_deactivate utility invokes a script called deactivate, if it exists, instead of removing the default set of symbolic links. The package author must make sure that the activate and deactivate scripts are self-consistent.
If the package provides a ROOT macro called <package>Logon.C, where <package> is the name of the package, then this macro will automatically be executed when ROOT starts up. This can be used to load a ROOT-enabled shared-object library provided by this package, for instance, as the Fr package does. Note that this automatic macro execution is a special feature of the LIGOtools version of ROOT.

The postinstall, activate and deactivate scripts are all executed with the version's directory as the current directory, and with the LIGOTOOLS environment variable set to the main LIGOtools directory.

Maintaining the LIGOtools Web Site

The web site is hosted by the LDAS software server, ldas-sw.ligo.caltech.edu, and the files are owned by user ldas. Everything is kept within the directory /ldcg_server/htdocs/ligotools. This directory contains a few html files, plus one subdirectory for each package, which in turn contains an index.html file plus a subdirectory for each version.

After copying a new distribution file to the web site, it is essential that you run the makelist script (in /export/ldcg_server/htdocs/ligotools). This script scans all of the distribution files and updates the file /export/ldcg_server/htdocs/ligotools/packages.txt, which is used by the ligotools_update script to determine when new packages/versions are available for download. If you forget to run the makelist script, then people are unlikely to notice that there is something new.

The makelist script assumes that distribution files are stored in the current directory structure. If this is expanded someday (e.g. by linking to packages stored at other sites), then the makelist script will have to be make smarter.