specplot_gallery

Read a list of SPEC data files (or directory(s) containing SPEC data files) and plot images of all scans. specplot_gallery will store these images in subdirectories of the given base directory (default: current directory) based on this structure:

{base directory}
   /{year}
      /{month}
         /{spec file name}
             /index.html
              s00001.png
              s00002.png

The year and month are taken from the SPEC data file when the data were collected. The plot names include the scan numbers padded with leading zeroes to five places (so the file names sort numerically).

The results will be shown as a WWW page (index.html) of thumbnail images and a separate list of any scans that could not generate plots. A reason will accompany these scans, as shown in the example.

How to use specplot_gallery: command line

Here is an example:

user@host ~$ specplot_gallery -d ./__demo__ ../spec2nexus/data/33bm_spec.dat

Example of specplot_gallery showing scans from test file 33bm_spec.dat.

Note that one of the scans could not be plotted. Looking at the data file, it shows there is no data to plot (this particular scan was aborted before any data was collected):

#C Wed Jun 16 19:00:10 2010.  Scan aborted after 0 points.

The last scan shown is from a hklmesh (2-D) scan. It is mostly a constant background level, thus the large black area.

Each of the plots in the web page can be enlarged (by clicking on it).

How to use specplot_gallery: periodic background task (cron)

This script could be called from a Linux background task scheduler (cron) entry. To add the entry, type the crontab -e command which opens the task list in a screen editor and add lines such as these to the file:

# every five minutes (generates no output from outer script)
0-59/5 * * * *  /path/to/specplot_gallery.py -d /web/page/dir /spec/data/file/dirs

If the specplot_gallery script is called too frequently and the list of plots to be generated is large enough, it is possible for more than one process to be running. In one extreme case, many processes were found running due to problems with the data files. To identify and stop all processes of this program, use this on the command line:

kill -9 `ps -ef | grep python | awk '/specplot_gallery.py/ {print $2}' -`

source code documentation

read a list of SPEC data files (or directories) and plot images of all scans

DirectoryNotFoundError	Exception: The requested directory does not exist
PlotSpecFileScans(filelist[, plotDir, ...])	read a SPEC data file and plot thumbnail images of all its scans
Cache_File_Mtime(base_dir)	Maintain a list of all known data file modification times.
datePath(date)	Convert the date into a path: yyyy/mm.
getSpecFileDate(specFile)	Return the #D date of the SPEC data file or None.
needToMakePlot(fullPlotFile, mtime_specFile)	Determine if a plot needs to be (re)made.
timestamp()	current time as yyyy-mm-dd hh:mm:ss
buildIndexHtml(specFile, plotted_scans, ...)	Build index.html content.
logger(message)	Log a message or report from this module.

RESULT

The images are stored in files within a directory structure that is organized chronologically, such as: yyyy/mm/spec_file/s1.svg. The root of the directory is either specified by the command line -d option or defaults to the current working directory. The yyyy/mm (year and month) are taken from the #D line of the SPEC data file. The spec_file is the file name with file extension and directory name removed. The image file names are derived from the scan numbers.

Linux CRON task

This script could be called from a cron entry, such as:

# every five minutes (generates no output from outer script)
0-59/5 * * * *  /some/directory/specplot_gallery.py -d /web/page/dir /spec/data/file/dir

If this script is called too frequently and the list of plots to be generated is large enough, it is possible for more than one process to be running. In one extreme case, many processes were found running due to problems with the data files. To identify and stop all processes of this program:

kill -9 `ps -ef | grep python | awk '/specplot_gallery.py/ {print $2}' -`

class spec2nexus.specplot_gallery.Cache_File_Mtime(base_dir)

Bases: object

Maintain a list of all known data file modification times.

Parameters:

base_dir (str) – name of base directory to store output image thumbnails

This list will allow the code to avoid unnecessary work reparsing and plotting of unchanged SPEC data files.

get(fname, default={'mtime': 0, 'size': 0})

Get the mtime cache entry for data file fname.

Parameters:

fname (str) – file name, already known to exist

Returns:

time (float) cached value of when fname was last modified or None if not known

read()

Read the cache from storage.

was_file_updated(fname)

Compare the mtime between disk and cache.gy

Parameters:

fname (str) – file name, already known to exist

Return bool:

True if file is newer than the cache (or new to the cache)

write()

Write the cache to storage.

exception spec2nexus.specplot_gallery.DirectoryNotFoundError

Bases: ValueError

Exception: The requested directory does not exist

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception spec2nexus.specplot_gallery.PathIsNotDirectoryError

Bases: ValueError

Exception: The path is not a directory

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class spec2nexus.specplot_gallery.PlotSpecFileScans(filelist, plotDir=None, reverse_chronological=False)

Bases: object

read a SPEC data file and plot thumbnail images of all its scans

Parameters:

• filelist ([str]) – list of SPEC data files to be checked

• plotDir (str) – name of base directory to store output image thumbnails

specFileUpdated(specFile)	Report if specFile has been updated.
plot_all_scans(specFile)	Plot all the recognized scans from file specFile.
getPlotDir(specFile)	Return the plot directory based on the specFile.
getBaseDir(basename, date)	Find the path based on the date in the spec file.
href_format(basePlotFile, altText)

getBaseDir(basename, date)

Find the path based on the date in the spec file.

getPlotDir(specFile)

Return the plot directory based on the specFile.

Parameters:

specFile (str) – name of SPEC data file (relative or absolute)

href_format(basePlotFile, altText)

plot_all_scans(specFile)

Plot all the recognized scans from file specFile.

specFileUpdated(specFile)

Report if specFile has been updated.

Return mtime cache entry (or None if not updated)

spec2nexus.specplot_gallery.buildIndexHtml(specFile, plotted_scans, problem_scans)

Build index.html content.

Parameters:

• specFile (str) – name of SPEC data file (relative or absolute)

• plotList ([str]) – list of HTML <a> elements, one for each plot image

spec2nexus.specplot_gallery.datePath(date)

Convert the date into a path: yyyy/mm.

Parameters:

date (str) – text date from SPEC file #D line: ‘Thu Jun 19 12:21:55 2014’

spec2nexus.specplot_gallery.developer()

Supply a file and a directory as command-line arguments to “paths”.

spec2nexus.specplot_gallery.getSpecFileDate(specFile)

Return the #D date of the SPEC data file or None.

Parameters:

specFile (str) – name of SPEC data file (relative or absolute)

spec2nexus.specplot_gallery.logger(message)

Log a message or report from this module.

Parameters:

message (str) – text to be logged

spec2nexus.specplot_gallery.main()

spec2nexus.specplot_gallery.needToMakePlot(fullPlotFile, mtime_specFile)

Determine if a plot needs to be (re)made. Use mtime as the basis.

Return bool:

True if plot should be made again

spec2nexus.specplot_gallery.timestamp()

current time as yyyy-mm-dd hh:mm:ss

Return str: