LDAS logo
TclDOC logo

The genericAPI.tcl Script

Modification Date: 11/26/2008

Table of Procedures

red ball Main
red ball addNewResourcesToFile
red ball anonFtpToplevel
red ball apiDirectory
red ball bCrypt
red ball bak
red ball bakDemangle
red ball bgLoop
red ball binaryDecrypt
red ball binaryEncrypt
red ball bootLock
red ball checkLeaks
red ball checkMySetup
red ball cloneProc
red ball cmdGet
red ball cmdSets
red ball cmonResource::deleteResourcesFromFile
red ball countChannels
red ball cppBase64Init
red ball cryptFileInPlace
red ball curlGetRemoteFileSize
red ball decryptFile
red ball destructElementWrap
red ball dropStatusPage
red ball dumpData
red ball dumpFile
red ball execscp
red ball execssh
red ball expandOpts
red ball fifo
red ball fileExists
red ball fileIsBeingWritten
red ball fileType
red ball fixUrlTarget
red ball formatAndProtocol
red ball formatConversion
red ball freeMemOnBox
red ball ftpPutLocal
red ball getApiOS
red ball getCurlUrl
red ball getCurlUrlCallback
red ball getIPAddress
red ball getMddTarget
red ball getSockData (reads data from a socket, usually
red ball getUrl
red ball gifBalls
red ball gridFtpToplevel
red ball ifConfig
red ball infoVars
red ball int2roman
red ball isBinary
red ball itemCount
red ball jobDirectory
red ball lastline
red ball leakLogger
red ball leaksSummary
red ball libstdcPlusPlus
red ball macroReturnMsg
red ball managerOutputUrl
red ball mapTCLVarToCVar
red ball mapTCLVarToCVar
red ball memFlag
red ball metaOpts
red ball myIP
red ball myName
red ball newhead
red ball newtail
red ball numRange
red ball outputFormat
red ball outputUrls
red ball outputUrlsBg
red ball packageReport
red ball parseURL
red ball pingAPI
red ball pongAPI
red ball popMsg
red ball portInfo
red ball procList
red ball procServer
red ball pubDirSetup
red ball publicFile
red ball putCurlUrl
red ball putUrl
red ball randomNumber
red ball realTimeRscValues
red ball relativeDirectory
red ball revArray
red ball roVar
red ball safeRxPat
red ball saveResourceToFile
red ball setAlertDebug
red ball setAlertDebugCB
red ball setLdasSystemName
red ball setResourceLimit
red ball shellPipe
red ball sleep
red ball sourceFile
red ball sourceRsc
red ball stats
red ball touch
red ball trace
red ball traceTimeout
red ball ucase
red ball unknown
red ball unpackTarball
red ball unwrapText
red ball url2file
red ball validFilename
red ball validProc
red ball validateEtcHosts
red ball varType
red ball when
red ball wrapText

# The Laser Interferometer Gravitational Observatory Data Analysis System genericAPI.tcl script.
This module sources the following sub-modules:

  1. stack.tcl (stack and queue manipulation functions)
  2. service.tcl (service availability and addressing)
  3. log.tcl (logging functions)
  4. cmd.tcl (command formatting functions)
  5. sock.tcl (socket communication functions)
  6. key.tcl (key generation and encryption functions)
  7. ilwd.tcl (ilwd text and object manipulation functions)
  8. timers.tcl (timing routines for benchmarking, etc.)
  9. ftp.tcl (ftp functions)
  10. queue.tcl (queue management functions)
  11. gpstime.tcl (gps time conversion routine)
  12. procfs.tcl (routines for parsing /proc file systems)
  13. smtp.tcl (routines implementing SMTP)
It is anticipated, and it is the criterion for their inclusion here, that all or most of these functions will be used in all or many of the other API's comprising the LIGO Data Analysis System.

Name: Main

Description:
Define the installation directory for the LDAS modules.
This is the "main" procedure for the
genericAPI.tcl module.
The local directory structure may be defined in any of the following ways:
  1. The environment variables LDASDIR, LDASHELPDIR, LDASLOGDIR, etc. can be set to the absolute path names.
  2. The variables LDAS, LDASHELP,LDASLOG, etc. can be set in the resource file to the absolute paths.
  3. Either LDASDIR or LDAS can be set, in which case LDASHELP and LDASLOG will be inferred to be the "help" and "log" directories immediately below it.
  4. If none of the variables is set then the current working directory will be taken as the top of the LDAS tree, and the help and log directories will be assumed to exist immediately below it.
Comments:
In any case where a directory location is inferred an announcement will be made to stderr describing the default action.
set ::RCS_ID_genericAPItcl { $Id: genericAPI.tcl,v 1.669 2008/05/29 23:43:36 mlei Exp $ }
set ::RCS_ID_genericAPItcl [ string trim $::RCS_ID_genericAPItcl "\$" ]
Prints a tcl proc callstack in stderr if resource is enabled
Must reboot API to take effect
if { [ info exist ::DEBUG_PROC ] && $::DEBUG_PROC } {
	catch { info proc *proc } err
	if	{ [ lsearch $err proc ] == -1 } {
    	rename proc tcl_proc
    	set ::native_proc tcl_proc
		puts stderr "tcl command 'proc' renamed to tcl_proc"
	}
	if	{ [ lsearch [ info command tcl_proc ] tcl_proc ] != -1 } {
		tcl_proc proc { args } {
			if 	{ [ lsearch [ info command puts ] puts ] != -1 } {
				set putcmd puts
			} 	elseif { [ lsearch [ info command tcl_puts ] tcl_puts ] != -1 } {
				set putcmd tcl_puts
			}	 else {
				set putcmd ::puts
			}
	
			;## no tcl_puts
			# $putcmd "args '$args'"
			set name [ lindex $args 0 ]
			set params [ lindex $args 1 ]
			set body [ lrange $args 2 end ]
			set body [ lindex $body 0 ]
			#$putcmd stderr "no renamed puts: name=$name, params='$params', body '$body' "
	
			set text "catch { info level -1 } caller\n$putcmd \"\[ clock seconds \]: $name called by \$caller\""
			set body "$text\n$body"
	
			#$putcmd "body '$body'"
	
			eval uplevel tcl_proc [ list $name [ list $params ] [ list $body ] ]
			# $putcmd "$name proc [ info proc $name ]"
		}
	}
} else {
	set ::native_proc proc
	# addLogEntry "tcl command 'proc' is unchanged" purple
}

§   §   §
package provide generic 1.0 set genericAPI TRUE package require -exact stack 1.0 package require -exact procfs 1.0 package require -exact smtp 1.0 package require -exact service 1.0 package require -exact log 1.0 package require -exact ilwd 1.0 package require -exact cmd 1.0 package require -exact sock 1.0 package require -exact stat 1.0 ;##package require -exact gui 1.0 package require -exact key 1.0 package require -exact queue 1.0 #package require FTP 1.3 package require -exact timers 1.0 package require http 2.0 package require base64 1.0 package require gpstime 1.0 package require RawGlobus 1.0 package require RawGlobusClient 1.0
§   §   §
Create custom versions of some internal procs set ::native_trace trace if { [ lsearch [ info proc setAlert ] setAlert ] == -1 } { ##------------------------------------------------------------------- ## Use 8.3 syntax to ensure compatability ##------------------------------------------------------------------- $::native_trace variable ::TID_FINISHED rw ::setAlertDebugCB } else { ::setAlertDebugCB }
§   §   §
if { [ lsearch [ info proc *trace ] tcl_trace ] == -1 } { rename trace tcl_trace set ::native_trace tcl_trace }
§   §   §
unset ::native_trace
Name: trace

Description:
Redefine trace to allow for additional debugging
Parameters: Usage:

Comments:

proc trace { args } {
    if { [ info exists ::DEBUG_TRACE ] \
	     && $::DEBUG_TRACE == 1 \
	     && [ info exists ::operator_socket ] } {
	set msg "'[ info level -1 ]' calling Trace: trace $args"
	addLogEntry $msg purple
    }
    if { [regexp {8.3} $::tcl_version]  \
	     && ( [llength $args] > 2 ) \
	     && ![regexp {variable|vdelete|vinfo} [lindex $args 0] ] \
	     && [regexp {variable} [lindex $args 1] ] } {
	## **************************************************************
	## Provide tcl 8.4 syntax for 8.3 interpreter
	## **************************************************************
	set majorcommand [lindex $args 0]
	set tuype [lindex $args 1]
	set name [lindex $args 2]
	set ops [list]
	set minorcommand [list]
	if { [llength $args] > 2 } {
	    foreach op [lindex $args 3] {
		switch -exact $op {
		    read {
			set ops "r$ops"
		    }
		    write {
			set ops "w$ops"
		    }
		    unset {
			set ops "u$ops"
		    }
		}
	    }
	    if { [string length $ops] <= 0 } {
		set $ops [lindex $args 3]
	    }
	
	    set minorcommand [list [lindex $args 4] ]
	}
	switch -exact $majorcommand {
	    add {
		uplevel tcl_trace variable $name $ops $minorcommand
	    }
	    remove {
		uplevel tcl_trace vdelete $name $ops $minorcommand
	    }
	    info {
		uplevel tcl_trace vinfo $name
	    }
	    default {
		uplevel tcl_trace $args
	    }
	}
	
    } else {
	uplevel tcl_trace $args
    }
}

§   §   §

Name: setAlertDebug

Description:
Redefine setAlert to allow for additional debugging Do NOT call this function as setAlertDebug as it will be renamed to setAlert once the genericAPI library has been loaded.

Parameters: Usage:

Comments:

proc setAlertDebug { args } {
    uplevel ldas_setAlert $args
    if { [ info exists ::DEBUG_TRACE ] \
	     && $::DEBUG_TRACE == 1 \
	     && [ info exists ::operator_socket ] } {
	set msg "'[ info level -1 ]' setAlert $args -- [getTIDDebugInfo [ lindex $args 0] ]"
	addLogEntry $msg purple
    }
}

§   §   §

Name: setAlertDebugCB

Description:
Helper function for setting up tracing of setAlert calls
Parameters: Usage:

Comments:

proc setAlertDebugCB { args } {
    if { [ lsearch [ info proc *setAlert ] ldas_setAlert ] == -1 } {
	trace remove variable ::TID_FINISHED [list read write] ::setAlertDebugCB
	rename setAlert ldas_setAlert
	rename setAlertDebug setAlert
    }
}

§   §   §

Name: checkMySetup

Description:
Checks the LDAS installation, performing the following steps:
  1. See if the machine knows it's name. if not, check for the existence of the variable $LOCALHOST. If the machine name cannot be determined, a Tcl exception is thrown.
    No attempt is made to defeat spoofing.
  2. Check to see if the LDAS toplevel directory has been declared, and whether the subdirectory locations are set. Set the toplevel to "pwd" if it's not found, and set the required subdirs to reasonable places beneath it.
  3. Check the declarations of required resource variables, throws an exception if any required value is not found.
  4. Finally, it checks for a valid filename for the local log file, and make up a likely one based on the name of the resource file if it needs to.
Usage:
       checkMySetup
Where "API" should be declared at the top of the resource
file by a line like:

set API usr
Comments:
Should be called after sourcing the
genericAPI.tcl and the local resource file.
proc checkMySetup { } {
     if { ! [ info exists ::API ] } {
        set msg    "The required variable \"API\" has not\n"
        append msg "been set, this probably means you are\n"
        append msg "trying to source the genericAPI.tcl\n"
        append msg "without providing a dummy .rsc file.\n"
        return -code error $msg
        }
     if { ! [ info exists ::env(HOST) ] } {
        if { ! [ info exists ::LOCALHOST ] } {
           set msg    "Your machine does not know it's own name,\n"
           append msg "and you have not set the variable\n"
           append msg "\"LOCALHOST\" to your machines' name.\n"
           append msg "please set the variable \"LOCALHOST\"\n"
           append msg "in your local LDAS${::API}.rsc file."
           return -code error $msg
        }
     }
     if { ! [ info exists ::LOCALHOST ] } {
        set ::LOCALHOST $::env(HOST)
     }
     roVar LOCALHOST
     set ::MY_IP [ myIP ]
     roVar MY_IP
     set ::BAD_WORDS (rename|open|socket|proc|file|exec|cd|pwd|load|exit|source|send)
     if { $::DONT_BLOCK_BAD_WORDS } {
        set ::BAD_WORDS bad_words_are_not_blocked_so_look_out
     }
     roVar ::BAD_WORDS
     foreach dir { {} LIB HELP LOG TMP ARC MACROS } {
        set dir "LDAS$dir"
        if { ! [ info exists ::$dir ] || \
             ! [ string length [ set ::$dir ] ] } {
           switch -exact -- $dir {
              LDAS {
                   set err "The variable ::LDAS seems to be undefined.\n"
                   append err "since this variable is set using the\n"
                   append err "autoconf macro \@prefix\@, this is a\n"
                   append err "serious problem.\n\n"
                   append err "Please examine the top section of the\n"
                   append err "managerAPI and LDASgwrap executable\n"
                   append err "scripts for the place where ::LDAS is\n"
                   append err "defined!"
                   return -code error $err
                   }
           LDASLIB {
                   set ::$dir [ file join $::LDAS lib ]
                   }
          LDASHELP {
                   set ::$dir [ file join $::LDAS help ]
                   }
           LDASLOG {
                   set ::$dir [ file join $::env(RUNDIR) logs ]
                   file mkdir [ set ::$dir ]
                   file attributes [ set ::$dir ] -permissions 0755
                   gifBalls   [ set ::$dir ]
                   }
           LDASTMP {
                   set ::$dir [ file join $::env(RUNDIR) tmp ]
                   file mkdir [ set ::$dir ]
                   file attributes [ set ::$dir ] -permissions 0770
                   }
           LDASARC {
                   set ::$dir [ file join $::LDASLOG archive ]
                   file mkdir [ set ::$dir ]
                   file attributes [ set ::$dir ] -permissions 0755
                   }
        LDASMACROS {
                   set ::$dir [ file join $::LDAS share ldas macros ]
                   }
               } ;## end switch
           if { $::DEBUG } {
              puts stderr "$dir set to [ set ::$dir ]"
           }
        } ;## end of if not info exists dir
        roVar $dir
     } ;## end foreach
     pubDirSetup
     if { ! [ info exists ::REQUIRED_VARIABLES ] } {
        set msg    "No list of required variables declared!"
        append msg "please put the list \"REQUIRED_VARIABLES\"\n"
        append msg "in your local LDAS${::API}.rsc file."
        return -code error $msg
     }
     foreach var $::REQUIRED_VARIABLES {
        if { ! [ info exists ::$var ] } {
           set msg    "Variable: $var not intitialised in\n"
           append msg "your local .rsc file.\n"
           append msg "This variable must be initialised."
           return -code error $msg
        }
     }
     ;## set standard ports for all API's
     set i 0
     foreach api $::API_LIST {
        array set ::$api "host [ set ::${::LDAS_SYSTEM}($api) ]"
        foreach sock { operator emergency data } {
           array set ::$api "$sock [ expr { $::BASEPORT + [ incr i ] } ]"
        }
    }
    unset i
    if { ! [ info exists ::LOCAL_LOG_FILE ] } {
       if { [ info exists ::LOCAL_LOG ] } {
          if { ! [ regexp {LDAS[a-z]+\.log} $::LOCAL_LOG ] } {
             set msg    "Malformed log file name:\n"
             append msg "$LOCAL_LOG.\n"
             append msg "Log file name must be of the form:\n"
             append msg "\"LDASapi.log\", where api is the\n"
             append msg "name of the current api, which might\n"
             append msg "be: user, mgr, frame, etc."
             return -code error $msg
          }
          if { $::DEBUG } {
             puts stderr "Variable LOCAL_LOG_FILE not set.\n"
             puts stderr "Setting it to $::LDASLOG/LDAS${::API}.log."
          }
       } else {
          set ::LOCAL_LOG "LDAS${::API}.log"
       }
       set ::LOCAL_LOG_FILE [ file join $::LDASLOG $::LOCAL_LOG ]
    }
}

§   §   §

Name: validateEtcHosts

Description:

Parameters: Usage:

Comments:

proc validateEtcHosts { args } {
     if { [ catch {
        set data [ dumpFile /etc/hosts ]
        set data [ split $data "\n" ]
        foreach line $data {
           if { [ regexp {^\s*127.0.0.1} $line ] } {
              break
           }
        }
        if { ! [ string equal localhost $::env(HOST) ] && \
               [ lsearch $line $::env(HOST) ] != -1 } {
           set subject "$::API API: $::env(HOST) aliased to loopback!"
           set msg    "/etc/hosts file declares $::env(HOST) to "
           append msg "be an alias for the loopback address.\n"
           append msg "if all API's in this LDAS system are running\n"
           append msg "on $::env(HOST) this is not a problem, but if\n"
           append msg "any API's run on OTHER MACHINES this is likely\n"
           append msg "to cause data socket communication problems!"
           return -code error $subject
        }
     } err ] } {
        puts stderr $subject
        addLogEntry "Subject: ${subject}; Body: $msg" email
     }
}

§   §   §

Name: setLdasSystemName

Description:
Populates the value of the ::LDAS_SYSTEM resource variable.

Parameters: Usage:

Comments:

proc setLdasSystemName { } {
     if { [ catch {
        if { [ file exists /etc/ldasname ] } {
           set ::LDAS_SYSTEM [ dumpFile /etc/ldasname ]
        } else {
           set ::LDAS_SYSTEM localhost
        }
        ;## convert trailing integers to roman numerals!
        if { [ regexp {(.+)(\d+)$} $::LDAS_SYSTEM -> name int ] } {
           set ::LDAS_SYSTEM $name[ int2roman $int ]
        }
        set ::LDAS_SYSTEM [ string trim $::LDAS_SYSTEM ]
        roVar LDAS_SYSTEM
        set ::RUNCODE [ string toupper $::LDAS_SYSTEM ]
        puts stderr "::LDAS_SYSTEM set to '$::LDAS_SYSTEM'"
        ;## turn ::${::API}_API_HOST resource variables
        ;## into the system api host list.
        set hosts [ info vars ::*_API_HOST ]
        foreach host $hosts {
           regexp {::([^_]+)} $host -> api
           set host [ set $host ]
           set api [ string tolower $api ]
           set ::${::LDAS_SYSTEM}($api) $host
        }
     } err ] } {
        return -code error "[ myName ]: $err"
     }
}

§   §   §

Name: pubDirSetup

Description:
Determine public ftp and http areas, providing defaults if none are provided, as possible.

Parameters: Usage:

Comments:

Helper function for checkMySetup
proc pubDirSetup { } {
     if { ! [ info exists  ::FTPDIR ] || \
          ! [ file exists $::FTPDIR ] } {
        set ::FTPDIR  [ anonFtpToplevel ]
        ;## anon ftp may be set up badly -- it happens!
        if { [ string length $::FTPDIR ] && \
           ! [ file exists $::FTPDIR ]   && \
             [ string equal manager $::API ] } {
           file mkdir $::FTPDIR
           file attributes $::FTPDIR -permissions 0755
           puts stderr "toplevel ftp directory created: '$::FTPDIR'"
        }
        if { [ string length $::FTPDIR ] && \
           ! [ file writable $::FTPDIR ] } {
           if { [ file writable \
                [ file join $::FTPDIR pub outgoing ] ] } {
              set ::FTPDIR [ file join $::FTPDIR pub outgoing ]
           } elseif { [ file writable \
                      [ file join $::FTPDIR pub ] ] } {
              set ::FTPDIR [ file join $::FTPDIR pub ]
           } else {
              set ::FTPDIR [ list ]
           }
        }
     }
     set rel [ list ]
     set cwd $::env(RUNDIR)
     ;## no browser access to cwd!
     if { ! [ file exists index.html ] } {
        set fid [ open index.html w 0444 ]
        puts $fid <HTML>
        close $fid
     }
     regsub $::WORKING_DIRECTORY_MOUNT_POINT $cwd {} cwd
     regsub [ anonFtpToplevel ] $::FTPDIR {} rel
     if { ! [ info exists  ::FTPURL ] || \
          ! [ file exists $::FTPURL ] } {
        set ::FTPURL  ftp://$::MY_IP$rel
     }
     if { ! [ info exists  ::HTTPURL ] || \
          ! [ file exists $::HTTPURL ] } {
        set ::HTTPURL http://${::MY_IP}${cwd}/jobs
     }
     if { ! [ info exists  ::HTTPDIR ] || \
          ! [ file exists $::HTTPDIR ] } {
        set ::HTTPDIR [ file join $cwd jobs ]
     }
     if { ! [ info exists  ::PUBDIR ] || \
          ! [ file exists $::PUBDIR ] } {
        set ::PUBDIR $::HTTPDIR
     }
     set fname [ file join $::PUBDIR .htaccess ]
     if { ! [ file exists $fname ] } {
        file mkdir $::PUBDIR
        file attributes $::PUBDIR -permissions 0775
        set fid [ open $fname w 0444 ]
        puts $fid "DefaultType application/octet-stream"
        close $fid
     }
     setLdasSystemName
     set ::GRIDFTPDIR [ gridFtpToplevel ]
     if { [ string length $::GRIDFTPDIR ] } {
        set ::GRIDFTPURL gridftp:$::GRIDFTPDIR
     } else {
        set ::GRIDFTPURL [ list ]
     }
     puts stderr "::FTPDIR set to '$::FTPDIR'"
     puts stderr "::FTPURL set to '$::FTPURL'"
     puts stderr "::HTTPDIR set to '$::HTTPDIR'"
     puts stderr "::HTTPURL set to '$::HTTPURL'"
     puts stderr "::GRIDFTPURL set to '$::GRIDFTPURL'"
     puts stderr "::GRIDFTPDIR set to '$::GRIDFTPDIR'"
     puts stderr "::PUBDIR set to '$::PUBDIR'"
}

§   §   §

Name: getIPAddress

Description:
Returns the IP address of a machine, given the machine name. Will work on any UNIX system. Will do a reverse if an IP address is given!
Usage:
      set ip_address [ getIPAddress $host|$ipaddress ]
Comments:
proc getIPAddress { host } {
     set retval [ list ]
     if { [ catch {
        set sid [ socket -async $host 22 ]
        after 100
        set data [ fconfigure $sid -peername ]
        close $sid
        foreach [ list ip hostname port ] $data { break }
        if { [ string equal $host $ip ] } {
           set retval $hostname
        } else {
           set retval $ip
        }
     } err ] } {
        catch { ::close $sid }
        puts stderr "getIPAddress: failed to connect to $host port 22: '$err'"
     }
     return $retval
}

§   §   §

Name: myIP

Description:
Returns the IP address of the host machine. The version which is commented out is the canonical version, which can, under rare circumstances take VERY long to return on error.
The new version relies on the existence of sshd on the machine, which is an LDAS requirement.

Parameters: Usage:
       set ::__myip [ myIP ]
Comments:
Very slow! Should be used to create a global variable as in the usage example.
proc myIP {} {
     if { [ catch {
        set ip 127.0.0.1
        set sid [ socket -async [ info hostname ] 22 ]
        set ip  [ lindex [ fconfigure $sid -sockname ] 0 ]
        ::close $sid
     } err ] } {
        catch { ::close $sid }
        puts stderr "myIP error: '$err' on port 22 (sshd). using 127.0.0.1"
     }
     return $ip
}

§   §   §

Name: ifConfig

Description:
Returns a list of lists consisting of the interface id string, the IP address for the interface, and the aliases for the IP address from /etc/hosts.

Parameters: Usage:
  Example output:

{lo0 127.0.0.1 localhost} {ge0 10.16.0.5 gateway} {hme0 131.215.115.248 {ldas-dev ldas-dev.ligo.caltech.edu loghost}}
Comments:
proc ifConfig { args } {
     if { [ catch {
        set interfaces [ list ]
        if { [ file executable /usr/sbin/ifconfig ] } {
           catch { ::exec /usr/sbin/ifconfig -a } data
        } elseif { [ file executable /sbin/ifconfig ] } {
           catch { ::exec /sbin/ifconfig -a } data
        } else {
           return -code error "can't find 'ifconfig' executable!"
        }
        set fid [ open /etc/hosts r ]
        set hostdata [ read $fid [ file size /etc/hosts ] ]
        ::close $fid
        foreach line [ split $hostdata "\n" ] {
           array set hosts \
              [ list [ lindex $line 0 ] [ lrange $line 1 end ] ]
        }
        foreach line [ split $data "\n" ] {
           regexp {^(lo |[a-z]+\d+)} $line -> if
           set if [ string trim $if ]
           if { [ regexp {^\s+inet\s+(?:addr:)?(\S+)} $line -> ip ] } {
              if { [ info exists hosts($ip) ] } {
                 lappend interfaces [ list $if $ip $hosts($ip) ]
              } else {
                 lappend interfaces [ list $if $ip dhcp ]
              }
           }
        }
     } err ] } {
        return -code error "[ myName ]: $err"
     }
     return $interfaces
}

§   §   §

Name: validProc

Description:
Determines if a procedure is valid in the context within which it is called. Just allows a clean test for usability.

Parameters: Usage:
      if { ! [ validProc "procname" ] } { complain }
Comments:
Best made use of in the negated sense as shown above.
proc validProc { { name * } } {
     set namesp [ namespace children ]
     ;## if the argument was namespace aware
     if { [ regexp {::} $name ] } {
        set namesp {}
        }
     ;## if the arg was of the form ::foo*
     if { [ regexp {^(::[^:]+)\*} $name -> tmp ] } {
        set name ${tmp}::*
        }
     ;## examine all namespaces visible locally
     foreach ns $namesp {
        if { ! [ regexp {::$} $ns ] } {
           set ns ${ns}::
           }
        if { [ llength [ info commands $ns$name ] ] } {
           return 1
           }
        }
     ;## if the argument was a non-namespace proc
     if { [ llength [ info commands $name ] ] } {
        return 1
        }
     return 0
}

§   §   §

Name: procList

Description:
Returns a list of all the procs in the scope of the caller.
Usage:
      set ${API}procs [ procList ]
Comments:
Must be called by the interpreter whose proc list you want. Useful for knowing which interpreter to send a request to when multiple interps are available.
proc procList { { globpat * } { level 1 } } {
     return [ uplevel $level info commands $globpat ]
}

§   §   §

Name: revArray

Description:
generates reverse lookup key/value pairs for an array.
Usage:
       array set rev_array [ revArray array_name ]
Comments:
Will cause the array in the caller to be supplemented to include reverse lookup values. I could also be called so as to return the reverse values to a different array, useful when the array contents are dynamic.
proc revArray { { name "" } } {
     if { ! [ array exists $name ] } {
        return -code error "No array named \"$name\" in scope."
        }
     if { [ catch {
        array set local_array [ uplevel [ array get $name ] ]
        } err ] } {
        return -code error $err
        }
     set reverse [ list ]
     foreach {name value} [ array get local_array ] {
     lappend reverse $value $name
     }
     return $reverse
}

§   §   §

Name: dumpFile

Description:
return the contents of a file in a form suitable for "more" or "less" etc.
Usage:
      set data [ dumpFile filename ]
Comments:
This is an efficient slurper of files.
proc dumpFile { { file "" } } {
     if { ! [ string length $file ] } {
        return {}
        }
     if { ! [ file exists $file ] } {
        return {}
        }
     if { [ catch { set fid [ open $file r ] } err ] } {
        return -code error $err
        }
     set size [ file size $file ]
     if { $size == 0 } {
        set size 100000
     }
     set data [ read $fid $size ]
     catch { ::close $fid }
     set data [ string trim $data ]
     return $data
}

§   §   §

Name: publicFile

Description:
Create a new file in the ::PUBDIR of the API according to the correct naming convention for the subdirectory and filename.
Returns the correct full path to the output file, which can be immediately used by outputUrls.
The idea is, that you can pass a URI to publicFile, and then pass the return value of publicFile and the URI to outputUrls and the local file will mirror to the remote location.
This is the canonical LDAS method for creating a public space file.

Parameters: Usage:

Comments:

Fails silently on error with a log entry.
First level backups have extension .bak, second level has extension .bak2.
proc publicFile { jobid fname contents { format binary } { comp none } } {
     set seqpt {}
     if { [ catch {
        regexp {\d+} $jobid job
        set jobid $::RUNCODE$job
        ;## remove all blank spaces from dir and file names
        ;## and replace with underscores.
        regsub -all -- {\s+} $fname {_} fname
        ;## find out as much about the object to be written as
        ;## possible
        foreach [ list type contents ] [ varType $contents ] { break }
        ;## because sometimes a bug makes us try to copy
        ;## the *root* directory!!
        if { [ regexp {directory} $type ] } {
           set err "attempt to copy directory: '$contents' "
           append err "made in file context."
           return -code error $err
        }
        ;## determine the correct file extension intelligently
        set extension [ file extension $fname ]
        if { ! [ string length $extension ] } {
           if { [ regexp {unknown} $type ] } {
              set extension .txt
           } elseif { [ regexp {ilwd} $type ] } {
              set extension .ilwd
           } elseif { [ regexp {ligo_lw} $type ] } {
              set extension .xml
           }
        }
        ;## create the fully qualified path of the output file
        set rootname  [ file rootname [ file tail $fname ] ]
        set fname $rootname$extension
        set dir [ jobDirectory $jobid ]
        set fname [ file join $dir $fname ]
        ;## don't overwrite existing files
        bak $fname
        ;## do the real work of writing the file
        if { [ regexp {ilwd pointer} $type ] } {
           set seqpt {}
           ilwd::write2disk $jobid $fname $contents $format $comp
        ;## this can only happen in the frame API
        } elseif { [ regexp {framecpp pointer} $type ] } {
           set seqpt {}
           frame::writeFile $jobid $fname $contents
        } elseif { [ regexp {file} $type ] } {
           file copy -force -- $contents $fname
        } else {
           set fid [ open $fname w 0664 ]
           puts $fid $contents
           close $fid
           unset fid
        }
        if { [ file exists $fname ] } {
           file attributes $fname -permissions 0664
        }
     } err ] } {
        if { [ info exists fid ] } {
           if { [ catch {
              close $fid
           } err2 ] } {
              set err "$err: $err2"
           }
        }
        return -code error "[ myName ]:$seqpt $err"
     }
     return $fname
}

§   §   §

Name: validFilename

Description:
Detects the use of disallowed characters in a filename.
Handles cases of root filenames, filenames with an extension, and full path prepended filenames.
Usage:
      if { [ validFilename filename ] } { do }
Comments:
Unix users will find things a bit draconian.
Not exhaustive. Simply a first line of defense.
proc validFilename { { filename "" } } {
     set flag 1
     if { [ regexp {[^a-zA-Z0-9\-\_\.\~]} $filename ] } {
        set flag 0 ;## invalid filename!
        }
     set flag ;## filename is ok!
}

§   §   §

Name: pingAPI

Description:
Ping a port and see if the associated process lives.
get a little mini-report and put it in the managers file at some interval. Generate an alarm if something is awry.
Usage:
      pingAPI $api
Comments:
This works without blocking on an error!
proc pingAPI { api } {
     set busy    0
     set sid    {}
     set retval {}
     if { [ catch {
        set sid [ sock::open $api emergency ]
        fconfigure $sid -blocking off
        set pingkey [ key::time ]
        __t::start $pingkey
        puts $sid "$::MGRKEY {\${::API}::reply \$cid \[ pongAPI \]}"
        flush $sid
        set i 0;
        while { 1 } {
           incr i 20
           ;## if we get a response we're done
           if { [ regexp {\d{9,10}} [ gets $sid ] ] } {
              break
           } else {
              ;## how long have we been waiting?
              if { $i < 1000 } {
                 after 20
              } else {
                 ;## too long
                 set busy 1
                 break
              }
           }
        }
        set tmark [ ::__t::mark $pingkey ]
        ::__t::cancel $pingkey
        set tmark [ format "%.3f" $tmark ]
        set retval "mgr -> $api -> mgr in $tmark"
     } err ] } {
        set retval "$api API unreachable! $err"
     }
     if { $busy } {
        set host [ set ::${::LDAS_SYSTEM}($api) ]
        set status [ sock::diagnostic $host ]
        foreach { flag msg } $status { break }
        if { $flag } {
           set retval "$api API unreachable! $msg"
        } else {
           if { [ info exists retval ] } {
              if { [ regexp {unreachable} $retval ] } {
              } else {
                 set retval "$api emergency port is open but unresponsive"
              }
           }
        }
     }
     catch { ::close $sid }
     return $retval
}

§   §   §

Name: pongAPI

Description:
Pong back local gpsTime value Usage:

Comments:

Currently deprecated
proc pongAPI {} {
     return [ gpsTime ]
}

§   §   §

Name: popMsg

Description:
General purpose error annunciator function.
If you are running Tcl w/o Tk the message will just be puts'd, otherwise it will be in a little window located relative to the parent window.
Usage:
       popMsg msg win {delay}
Comments:
Note that to set a delay time different from the default of 2.5 seconds it is necessary to pass a "win" parameter, which may be "".
Technically this is lower priority than any of the logging functions, since no long entry is made.
proc popMsg { { msg "" } { win "" } { delay 2500 } } {
     global window; set window $win
     catch { [ winfo ] } errmsg
     if { [ string match invalid* $errmsg ] } {
        puts stderr $msg
        set msg {}
        }
     if { [ string length $msg ] } {
        destroy $win.mess
        frame   $win.mess       -class message \
                                -borderwidth 4 \
                                -relief raised
        label   $win.mess.label -foreground red \
                                -text $msg
        pack $win.mess.label -padx 12 -pady 12
        catch { ;## bury error if $win == ""
        place $win.mess -rely .1 -relx .25 -in $win
        }
        after $delay {
              destroy $window.mess
              }
        }
     return {}
}

§   §   §

Name: sourceRsc

Description:
Sources the LDASapi.rsc resource file in global context.
Usage:

Comments:

First tries to source the resource file in the current working directory. If it doesn't find it it will try to source the default resource file in $::LDAS/bin, which is probably NOT what you want!
proc sourceRsc { } {
     if { [ catch {
        set rscfile [ file join $::env(RUNDIR) LDASapi.rsc ]
        if { [ file exists $rscfile ] } {
           uplevel source $rscfile
        } else {
           uplevel source [ file join $::LDAS bin LDASapi.rsc ]
        }
        } err ] } {
        return -code error "Error sourcing LDASapi.rsc: $err"
        }
}

§   §   §

Name: sourceFile

Description:
Source a file. Limits the range of sourceable files to those found in the LDAS installation directory tree.
Usage:
      sourceFile filename subdir
Comments:
Just encapsulates a file sourcing function which has the "look and feel of the LDAS API's.
proc sourceFile { { filename "" } { subdir "" } } {
     if { ! [ string length $filename ] } {
        return -code error "sourceFile: filename not given"
        }
     if { [ string length $subdir ] } {
        set filename [ file join $subdir $filename ]
        }
     if { ! [ regexp $::LDAS $filename ] } {
        set filename [ file join $LDAS $filename ]
        }
     if { [ file exists $filename ] } {
        uplevel source $filename
        } else {
        set msg    "Tried to source:\n$filename\n"
        append msg "File not found."
        return -code error "sourceFile:\n$msg"
        }
     return {}
}

§   §   §

Name: myName

Description:
Returns the name of the proc that calls it.
Usage:
      set myname [ myName ]
Comments:
This is the canonical form for getting a procs name.
The utility of this is that a proc may be dynamically named, and the name managed much more easily this way than by trying to stack up and unstack proc names.
Note that level 0 is the context of myName, level -1 is the context of the caller. In general, you will not mess with the level. Though you could get the name of the proc that called the caller with -2, and so forth.
proc myName { { level "-1" } } {
     if { $level > 0 } {
        return -code error "myName: called with level > 0 ($level)."
     }
     if { [ catch {
        set name [ lindex [ info level $level ] 0 ]
     } err ] } {
        set name $::API
     }
     set name
}

§   §   §

Name: randomNumber

Description:
Implements a very long period uniform distribution random number generator. Returns a 31 bit random positive integer.
Usage:
      set number [ randomNumber (seed) ]
Where seed is a positive integer 0 < seed < 2147483648
and the value returned is likewise.

Comments:
Copyright 1995 by Roger E. Critchlow Jr., San Francisco, California.
All rights reserved. Fair use permitted. Caveat emptor.
The generator is one George Marsaglia, geo@stat.fsu.edu, calls the Mother of All Random Number Generators.
Modified by Philip S. Ehrens at LIGO 98.12.17
namespace eval random {
set a1 { 1941 1860 1812 1776 1492 1215 1066 12013 };
set a2 { 1111 2222 3333 4444 5555 6666 7777   827 };
set m1 { 30903 4817 23871 16840 7656 24290 24514 15657 19102 };
set m2 { 30903 4817 23871 16840 7656 24290 24514 15657 19102 };
}

§   §   §
proc random::rand16 { a m } {
    set n [ expr {
        [ lindex $m 0 ] +
        [ lindex $a 0 ] * [ lindex $m 1 ] +
        [ lindex $a 1 ] * [ lindex $m 2 ] +
        [ lindex $a 2 ] * [ lindex $m 3 ] +
        [ lindex $a 3 ] * [ lindex $m 4 ] +
        [ lindex $a 4 ] * [ lindex $m 5 ] +
        [ lindex $a 5 ] * [ lindex $m 6 ] +
        [ lindex $a 6 ] * [ lindex $m 7 ] +
        [ lindex $a 7 ] * [ lindex $m 8 ] }];
    return [ concat [ expr { $n >> 16 } ] [ expr { $n & 0xFFFF } ] [ lrange $m 1 7 ] ];
}

§   §   §
proc random::srand16 { seed } {
    set n1 [ expr { $seed & 0xFFFF } ];
    set n2 [ expr { $seed & 0x7FFFFFFF } ];
    set n2 [ expr { 30903 * $n1 + ($n2 >> 16) } ];
    set n1 [ expr { $n2 & 0xFFFF } ];
    set m  [ expr { $n1 & 0x7FFF } ];
    foreach i { 1 2 3 4 5 6 7 8 } {
      set n2 [ expr { 30903 * $n1 + ($n2 >> 16) } ];
      set n1 [ expr { $n2 & 0xFFFF } ];
      lappend m $n1;
          }
    return $m;
}

§   §   §
;## Externals
proc srandomNumber { seed } {
    set random::m1 [ random::srand16 $seed ];
    set random::m2 [ random::srand16 [ expr { 4321+$seed } ] ];
    return {};
}

§   §   §
proc randomNumber { { seed 0 } } {
    if { $seed } {
       srandomNumber [ expr { int ($seed) } ]
       }
    set random::m1 [ random::rand16 $random::a1 $random::m1 ];
    set random::m2 [ random::rand16 $random::a2 $random::m2 ];
    return [expr (( [ lindex $random::m1 1 ] << 16) + [ lindex $random::m2 1 ]) & 0xFFFFFFF ];
}

§   §   §

Name: bgLoop

Description:
Start asynchronous looping jobs. Jobs are ended by setting ::bg::jobs($name,run) to 0.
Usage:
       start: bgLoop $name $code $delay
        stop: set ::bg::jobs($name,run) 0
Comments:
Since multiple processes with the same name CAN be started, a unique name should be chosen for each process.
Process name bookkeeping must be done by the caller.
the job code MUST NOT BLOCK, and should never return explicitly.
proc bgLoop { { name NULL } { code "" } { delay 2 } } {
     if { ! [ llength [ namespace children :: bg ] ] } {
        namespace eval bg {}
        set ::bg::iterator 0
     }
     incr ::bg::iterator
     ;## register a new job if it has valid args
     if { ! [ string equal NULL $name ]      && \
            [ string length [ join $code ] ] } {
        set ::bg::jobs($name,run)   1
        set ::bg::jobs($name,code)  $code
        set ::bg::jobs($name,delay) $delay
        addLogEntry "Looping process $name started"
     }
     if { [ info exists ::bg::after ] && \
          [ lsearch [ after info ] $::bg::after ] != -1 } {
        after cancel $::bg::after
     }
     if { [ string equal NULL $name ] } {
        set dt 0
        foreach job [ array names ::bg::jobs *,run ] {
           set job [ lindex [ split $job , ] 0 ]
           if { [ string equal NULL $job ] || \
                [ string equal -1 $::bg::jobs($job,run) ] } {
              continue
           }
           ;## if the run flag == 0, unregister the job
           if { [ string equal 0 $::bg::jobs($job,run) ] } {
              foreach item [ array names ::bg::jobs $job,* ] {
                 unset ::bg::jobs($item)
              }
              addLogEntry "Looping process $job terminated"
              continue
           }
           ;## otherwise, eval!
           if { ! ($::bg::iterator % $::bg::jobs($job,delay)) } {
              set ts [ clock clicks -milliseconds ]
              if { [ catch {
                 eval $::bg::jobs($job,code)
              } err ] } {
                 set ::bg::jobs($job,run) 0
                 addLogEntry "$err ($::bg::jobs($job,code))" email
              }
              set te [ clock clicks -milliseconds ]
              set td [ expr $te - $ts ]
              set dt [ expr $dt + $td ]
              lappend data [ list $job $td ]
           }
        }
        ;## produce a timing report if required
        if { ($dt > 1000) && [ info exists ::PROFILE_BGLOOP ] && \
             [ string equal 1 $::PROFILE_BGLOOP ] } {
           addLogEntry "runtime per iteration: $dt ms ($data)" blue
        }
        set ::bg::after [ after 1000 bgLoop ]
     } else {
        ;## we are running the code block for the first time,
        ;## so we eval NOW.
        if { [ catch {
           set retval [ eval $::bg::jobs($name,code) ]
        } err ] } {
           if { [ info exists job ] } {
              set ::bg::jobs($job,run) 0
              addLogEntry "$err ($::bg::jobs($job,code))" email
           } else {
              addLogEntry "$err ('job' not defined)" email
           }
           set ::bg::after [ after 1000 bgLoop ]
           return -code error $err
        }
        set ::bg::after [ after 1000 bgLoop ]
        return $retval
     }
}

§   §   §

Name: anonFtpToplevel

Description:
Retrieve the toplevel anonymous ftp directory from the /etc/passwd file.

Parameters: Usage:

Comments:

proc anonFtpToplevel { } {
     set data [ list ]
     if { [ file readable /etc/passwd ] } {
        set fid [ open /etc/passwd r ]
        set data [ read $fid ]
        close $fid
     } else {
        return -code error "[ myName ]: /etc/passwd not readable"
     }
     foreach line [ split $data "\n" ] {
        if { [ regexp {^ftp} $line ] } {
           set tmp [ split $line ":" ]
           set dir [ lindex $tmp end-1 ]
           return $dir
        }
     }
     puts stderr "[ myName ]: no anonymous ftp on this system"
     return {}
}

§   §   §

Name: gridFtpToplevel

Description:
If a grid user exists, returns the home directory trailing slash is usually NOT attached.

Parameters: Usage:

Comments:

proc gridFtpToplevel {  } {
     set data [ list ]
     set dir  [ list ]
     if { [ file readable /etc/passwd ] } {
        set fid [ open /etc/passwd r ]
        set data [ read $fid ]
        close $fid
     } else {
        return -code error "[ myName ]: /etc/passwd not readable"
     }
     foreach line [ split $data "\n" ] {
        if { [ regexp {^grid} $line ] } {
           set tmp [ split $line ":" ]
           set dir [ lindex $tmp end-1 ]
           if { [ string length $dir ] && \
                [ info exists ::GRID_FTP_WRITABLE_SUBDIRECTORY ] } {
              set subdir $::GRID_FTP_WRITABLE_SUBDIRECTORY
              set subdir [ string trim $subdir / ]
              set dir $dir/$subdir
           }
           break
        }
     }
     if { ! [ string length $dir ] } {
        set msg "no grid ftp user found on this system.\n"
        append msg "this is ok if this system is not\n"
        append msg "supposed to be supporting grid ftp.\n"
        append msg "this error will not adversely affect\n"
        append msg "the running of LDAS unless requests\n"
        append msg "for grid ftp data transfer are made."
     } elseif { ! [ file exists $dir ] } {
        set msg "[ myName ]: grid ftp directory: '$dir' "
        append msg "not found.\nare you sure it is visible "
        append msg "from the host $::env(HOST) ?"
     } elseif { ! [ file isdirectory $dir ] } {
        set msg "[ myName ]: grid ftp directory: '$dir' "
        append msg "is not a directory."
     } elseif { ! [ file writable $dir ] } {
        set msg "[ myName ]: grid ftp directory: '$dir' "
        append msg "is not writable by user $::env(USER)"
     }
     if { [ string equal manager $::API ] && [ info exists msg ] } {
        puts stderr $msg
        set pre "$::LDAS_SYSTEM ${::API}API"
        set subject "$pre gridftp config error!"
        set msg "$pre $msg"
        addLogEntry "Subject: ${subject}; Body: $msg" email
     }
     return $dir
}

§   §   §

Name: expandOpts

Description:
Expands procedure options. If a procedure predeclares a list of arguments like this:
set opts { -user {} -host {} -command {} } then if the procedure is declared like this:
proc doThis { { args "" } } { ... } the procedure can be called like this:
doThis -u myname -h machine.domain.com -c die!
then doThis can call expandOpts, and the single letter form of the options will be expanded and used to modify the options to the procedure "in place".
Usage:
       array set opts [ expandOpts [opts] ]
     Where: opts is optional, defaulting to "opts".

Comments:
Uses an expensive but thorough "backing-up" regex pattern.
A command line *option* may NOT begin with a number.
An ambiguous option (one that matches more than one default) will throw an exception.
proc expandOpts { { opts "opts" } } {
     set options [ list ]
     set matches [ list ]
     ;## get inputs and massage them into a
     ;## well-formed list
     set inputs [ uplevel set args ]
     regsub -all -- {\s+} $inputs { } inputs
     regexp {^\{(.+)\}$}    $inputs  -> inputs
     ;## get defaults and do the same
     set defaults [ uplevel subst \$$opts ]
     regsub -all -- {\s+} $defaults { } defaults
     ;## and GO! (trimming loose spaces from values)
     foreach { opt def } $defaults {
        set opt     [ string trim $opt  ]
        set def     [ string trim $def  ]
        set matched 0
        foreach { name val } $inputs {
           set name [ string trim $name ]
           regsub -- {[-]+} $name "-" name
           set val  [ string trim $val  ]
           if { [ regexp -nocase -- ^$name $opt ] } {
              ;## next test fails if ambiguous item is last
              ;## option on command line...
              if { [ lsearch -exact $matches $name ] > -1 } {
                 return -code error "[ myName ]: ambiguous option: \"$name\""
              }
              lappend matches $name
              set options [ concat $options [ list $opt $val ] ]
              set matched 1
              break
           }
        }
        if { $matched == 0 } {
           set options    [ concat $options [ list $opt $def ] ]
        }
     }
     return $options
}

§   §   §

Name: metaOpts

Description:
Make optional arguments available to the API's, expanding them and putting them in a global array named after the unique jobid.
Note that the list of meta options is strictly limited to a very few useful arguments. The argument list is shared between the API's. User info is maintained at the manager.
After this proc sets the jobid array, the options are available to the eval'd code block in the API.
Usage:

Comments:

This will be called by the operator handler of the API.
proc metaOpts { args } {
     if { [ catch {
        set opts [ uplevel set opts ]
        array set tmp [ expandOpts ]
        set jobid $tmp(-jobid)
        #catch { unset ::$jobid }
        array set ::$jobid [ array get tmp ]
     } err ] } {
        return -code error "[ myName ]: $err"
     }
}

§   §   §

Name: traceTimeout

Description:
Will put a trace on a variable. If the trigger is not set within a predetermined timeout period, the trace will be removed.

Parameters: Usage:
       traceTimeout vname doThis [ timeout ]
the argument list for command must end with "args"
to eat the arguments added by trace.

it is better for varname to be a unique name.
Comments:
"cmd" will receive three arguments from the trace, vname, {}, and the new value of $vname. These arguments will be appended to the argument list of cmd.
See the Tcl man page for "trace".
proc traceTimeout { { vname "" } { cmd "" } { timeout 10 } } {
     if { ! [ string length $cmd ] } {
        return {}
        }