Historical notes on using DataViewer and
Diagnostic Test Tools (version 1.2 and earlier)
with data on disk

Dataviewer, and the Diagnostic Test Tools suite, were designed to get their data directly from a server using LIGO's "NDS" (Network Data Server) protocol. Neither one has the ability to read data from disk. However, Daniel Sigg has implemented a scheme to allow data to be read from disk, by starting up a "mini-NDS" process in the background in such cases. In the future, one or both programs will be upgraded to be able to read conveniently from disk.

The "mini-NDS" is really just a special version of the LIGO "daqd" data acquisition program, using only its playback capabilities. This scheme requires a rather specific organization for the data on disk: it must be stored in frame files, each containing one second of data, and with the GPS time indicated in the filename, e.g. "H-659554975.F". (Note that names containing other numbers, e.g. "NORMAL1234-H-659554975.F", are not allowed. Also, it seems that the suffix cannot be arbitrarily long--for instance, files ending in .NORMAL don't work.) The files must be stored in directories with a specific naming convention: the first one must have a name of the form something.0, and if there are more in the sequence, they must be named something.1, something.2, etc. (Here, "something" can be replaced by any name you want.) For example, in my /home/pshawhan/data directory I might create the subdirectories e2data.0, e2data.1, and e2data.2, and put the files
H-657760000.F, H-657760001.F, ... through H-657760999.F into e2data.0 ;
H-657761000.F through H-657761999.F into e2data.1 ; and
H-657762000.F through H-657762999.F into e2data.2.

The reduced data sets (RDS's) written to disk during the engineering runs (at least through E3) are stored on fortress (at LHO) and decatur (at LLO) on the /export/raid* disks according to this convention, so if you are logged into one of those machines and want to look at the RDS data, skip the following two paragraphs and go directing to starting up dtt.

Note added following the E7 run: Beginning with version 4.3 of the dataflow package, you can use the getFrames utility to retrieve remote data and write it to disk as 1-second-long frame files in a subdirectory, by using the '-1' flag. This saves you the step of splitting up a big frame file as described in the following paragraph.

If you have a single frame file containing many one-second frames (for instance, if you used getFrames to retrieve a stretch of data from the central archive or from some other LARS server), then there is a utility called FrSplit which splits the data into individual one-second frame files. For example, if the file is called bigfile.F and contains 3000 frames, then:
  FrSplit -i bigfile.F -o datadir -n 1000
creates three directories, datadir.0, datadir.1, and datadir.2, and puts 1000 one-second frame files with appropriate names (H-667560000.F, H-667560001.F, etc.) into each directory. This is the proper organization for the next step.

To analyze this data with Dataviewer or with one of the DTT utilities (e.g. Fourier Tools), do the following:

• Type dtt to bring up the DTT main menu window.
• Under the "File" menu, choose "Setup...".
• Click on the "File" button.
• In the dialog box, navigate to find the first directory containing the data files -- in this case, /home/pshawhan/data/e2data.0 . Don't double-click on this directory's folder icon, just single-click to select it and then press the "Open" button". The dialog box will disappear, and back in the "Configuration" window the "File:" radiobutton should now be selected, and the name of the first data file should appear to the right of it.
• Click on the "Ok" button.
• Now you can click on one of the buttons in the DTT main menu window, e.g. "Data Viewer" or "Fourier Tools". This starts up a "mini-NDS" process which knows about the files in the directory you specified (e2data.0) plus any other directories in the same sequence (e2data.1, e2data.2), and then starts up the client process (Dataviewer or Fourier Tools) connected to this mini-NDS. This client process can access only the data from this particular mini-NDS; to access other data, you will need to repeat the procedure above and start up a new {mini-NDS,client} pair.

For Dataviewer, use the "Playback" menu choice to view the data. Here are a few things to remember:

• Remember to set the number of channels, which is 16 by default.
• On the Playback Control Panel, remember to select "Full Data" (it is set to "Minute Trend" by default, which the mini-NDS cannot provide).
• By default, a time range is entered as a stop time plus a duration. There is a pulldown menu to let you change this to start time plus duration.
• If want to enter the time range as GPS, you need to remember to click on the "GPS" radiobutton.
• Click to select the "Auto Setting" option, if you want auto-ranging along the vertical axes of the plots.
• If you need to manually set the vertical range of an Xmgr plot, you can do this with the "World scaling" choice under the "Plot" menu.

For DTT clients (Fourier Tools and Triggered Time Series), choose the desired channels and parameters and enter the start time, in GPS seconds or UTC date/time, for the data interval you want to look at. Also remember to click on the circle next to the time you specify. (If you get the status message "Test timed-out", you probably forgot to do this.) Finally, be sure to hit Return after you enter the time--otherwise, it may not register! (If you get a "Data receiving error" or "Synchronization error", it is probably because the frame file directory you are currently pointing to does not contain data for the time you specified, or else you forgot to hit Return to register the time you entered.)