\documentclass[10pt]{article}
\usepackage{epsfig, multicol}
%\include{latex}
\begin{document}
% This is mondo annoying: you have to put the space in or it will
% butt it up against the next word in a sentence, but if it is at
% the end of a sentence then you don't want the space.
%\newcommand{\dz}{D\O\ }
\newcommand{\bs}{$\backslash$}
% I think these sorts of commands can be used to mess with the pic's
% Right now I can only get one pic-per-page
%\renewcommand\floatpagefraction{1}
%\renewcommand\textfraction{0}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% FEATURES AND COMPONENTS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{flushright}
D\O\ Note 4052 \\
version 1.1 \\
21 November 2002
\end{flushright}
\bigskip
\bigskip
\bigskip
\begin{center}
{\LARGE \bf \sc The Calibration Manager} \\
\medskip
{\large \sc A Web Interface for Monitoring the D\O\ Pre-Shower
Calibration Database}
\end{center}
\bigskip
\begin{center}
Jeremy Herr (herrj@umich.edu), Homer Neal (haneal@umich.edu), Eric
Myers (myers@vassar.edu), University of Michigan Physics Dept.
\end{center}
\bigskip
\bigskip
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% INTRODUCTION
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\setcounter{section}{0}
\section{Introduction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% PREMISE
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Premise}
In the D\O\ experiment at Fermilab, each subdetector provides a means
for collecting calibration information. It is useful for physicists
concerned with calibration issues to know the value of pedestals, mips
and gains of each channel and each run. For the Central Pre-shower
subdetector, a database has been designed to store calibration data,
which will be described in detail in a separate document. The
Calibration Manager is an application designed to provide web-based
access to the Central Pre-shower calibration database, and because of
its flexibility, may also be used for other subdetectors' calibration
databases.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% DESIGN GOALS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Design Goals}
The purpose of the Calibration Manager is to provide a convenient
web-based interface for Pre-Shower Calibration Users to access the
Calibration Databases at Fermilab D\O. Armed with only an internet
connection and a web browser, any interested party may read
calibration data and view relevant graphs, and authorized physicists
may also use the same interface to manage the database, making
modifications and updating it.
Some desired features that motivated the design of the Calibration Manager
are for users to be able to:
\begin{itemize}
\item Examine calibration database tables
\item Plot distributions of any numeric calibration field
\begin{itemize}
\item as a function of run id
\item as a function of time {\em t} (when {\em t} is available)
\item as a frequency distribution of occurrence of field values within
given binning
\item as a scatterplot of two fields
\item classed by a single channel or by specified groups of channels
\end{itemize}
\item Conduct comparative analysis of current run with previous
specified runs, with a template, or with a running average
distribution
\item Provide Access to a SQL prompt via a web-interface (with and
without restrictions, depending on user's database permissions)
\end{itemize}
\begin{figure}[ht]
\centering
\epsfig{figure=VDoverview.eps,width=10cm}
\caption{``General Design Overview''}
\label{VDoverview}
\end{figure}
In addition to providing graphs of calibration data, the Calibration
Manager also serves as a web-based interface to other applications. A
separate calibration tool called ``TreeBuilder'' is an application
that uses sophisticated logic to find which calibration runs are
associated with which data runs. The Calibration Manager can be used
to verify that the TreeBuilder logic is functioning properly, run
other applications that perform advanced summary functions, and even
make alterations to the database. The Calibration Manager logs all
modifications to the database, and this logbook may be viewed by
authorized users. Thus by calling up the Calibration Manager's web
pages in his or her browser, the user can conveniently access all
functionality related to the calibration database. {\bf Figure
\ref{VDoverview}} shows an overview of the design.
Some advantages over existing database client software include:
\begin{itemize}
\item Availability to anyone with a browser and internet connection
\item No need for users to install client software (database client
resides on the web server)
\item Graphical User Interface is intuitive and self-explanatory
\item Central location for all database access needs
\item Access to other tools, including ROOT and Python scripts,
for performing advanced functions
\end{itemize}
The Calibration Manager lives on a web server ({\bf Figure
\ref{servers}}), which acts as an intermediate step between the end
users and the data server at Fermilab. Users request information by
filling out web forms and clicking on buttons, and the web server in
turn communicates with and fetches data from the database, returning
it to the end user in a user-friendly format.
The web server can serve multiple users at once, but it acts as a
single client when accessing the database. This setup circumvents the
issue of licensing for multiple Oracle Clients, since only one Oracle
Client is needed, which lives on the web server.
\begin{figure}[ht]
\centering
\epsfig{figure=servers.eps,width=10cm}
\caption{``web server / data server relationship''}
\label{servers}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% OVERVIEW OF WEB PAGES
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\bigskip
\section{Overview of Web Pages}
The Calibration Manager is accessed through a collection of dynamically
created web pages. In this section we describe the functionality and
use of these web pages, and include screen shots of the actual web pages
for clarity.
\subsection{The Welcome Page}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of Welcome page
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSwelcome.eps,width=8cm}
\caption{``Welcome Page screen shot''}
\label{SSwelcome}
\end{figure}
The starting point is the welcome page
(currently located at \\
\verb+http://circe.physics.lsa.umich.edu+),
whose purpose is to present the user with all available functions.
These functions have been broken down into three categories:
Selection, Browsing and Managing. All users may freely access the first
two categories, but an authorized login is required for management functions
(these are functions that modify data in the database).
{\bf Figure \ref{SSwelcome}} shows a screen shot of the welcome page.
Anonymous users (those who have not logged in using their D\O\ database password)
will see the words ``anonymous login'' in the upper-left corner,
which will be replaced by the user name upon authorized login
(click on {\tt SECURE DATABASE LOGIN} to authenticate).
The user must have a database account to use any of the manager functions.
\begin{itemize}
\item
{\bf \em DATABASE SELECTION} \\
The user should first select which database (offline development,
integration or production) he or she would like to look at, as well as
which subdetector (CPS, FPS or CFT), then click on the {\tt SET}
button. The page will reload, and the updated choices will be
displayed in the upper right corner of the welcome page. Once
selected, the subdetector and database settings are passed along from
page to page as hidden variables (there is no need to log in or accept
any cookies). Whenever the Calibration Manager is used to access a
D\O\ database, the user's current selections are assumed. They can be
changed at any time by returning to the welcome page and choosing
again.
\item
{\bf \em BROWSING THE DATABASE} \\
The second group of items in the welcome page contains database browsing
functions, which are available to all users. These functions
are restricted to reading data, so there is no danger of
careless or malicious users damaging the
contents of the database.
The following list of options is expounded upon in greater detail in the next
sections of this document.
\begin{itemize}
\item
{\bf FastGraph} \\
View complex ROOT graphs and analyses of pre-defined data sets.
\item
{\bf Plotting Portal} \\
View several simple graphs at once, side by side.
\item
{\bf WebSQL} \\
A handy command-line interface for the user who knows SQL.
\item
{\bf Listing} \\
Get a summary of calibration runs (generated by a Python script).
\item
{\bf Entity-Relationship (E-R) Diagram} \\
Click here to pop-up a window with an E-R diagram of the current
database, which shows a summary of the tables and how they are related
to each other.
\end{itemize}
\item
{\bf \em MANAGING THE DATABASE} \\
All of the functions under this heading require authorized login, which
currently means that the user must have a D\O\ Oracle Database account
to use them. Attempts to use them anonymously will result in failure.
See the section on Security below for an explanation of how authentication
and authorization are handled. A list of management functions follows,
and is given in greater detail in later sections.
\begin{itemize}
\item
{\bf Secure Database Login} \\
Submit your Oracle database password via an encrypted (SSL) link.
\item
{\bf TreeBuilder} \\
Runs the ``TreeBuilder'' routine from the package {\sc cps\_calibration} to
collect together separate calibration runs for pedestals, MIPS, and
gains for different parts of the detector to create a complete set of
calibration data for all channels.
\item
{\bf Delete Runs} \\
Runs a Python script for deleting specified calibration runs from the
database.
\item
{\bf Delete Calibrations} \\
Deletes a specified calibration from the database, along with
all the ``subsets'' that tie it to a calibration collection.
\item
{\bf On- to Offline Transfer} \\
Runs the transfer tool script {\tt calDbTransfer.py} from the package
{\sc onl\_calDbTransfer} to transfer calibration data from the on-line
calibration database to the off-line calibration database.
\item
{\bf LogBook} \\
View modifications made to the calibration database, sort and search
based on various criteria.
\item
{\bf Submit Graph Query Object (GQO)} \\
Create a new Graph Query Object using a simple HTML form, enabling
one-click SQL queries and graphs.
\end{itemize}
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% WALKTHROUGH OF EACH COMPONENT
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% FASTGRAPH
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% This page provides a list of pre-defined database queries for commonly
% used functions. For example, the user need only enter a desired
% channel and click a button to see, the pedestal values sorted by run
% number for that particular channel. Using the Graph Query Object
% submission page, authorized users can easily create new queries and
% graphs to be displayed on the FastGraph page. MISWEB is another
% package that can be used to make web-based pre-defined database
% queries, but currently without any graphing capability.
%
%I'm not sure where this section was supposed to go. I am thinking
%that I meant to get rid of it but didn't.
\subsection{The FastGraph Page}
The FastGraph page presents the user with a list of links
corresponding to database queries. The user might input qualifying
information, such as channel number or a choice between
pedestals/mips/gains, then click on the button and get a ROOT graph.
The purpose of FastGraph is to make commonly desired database queries
and graphs very easy to access, requiring no knowledge of SQL or ROOT.
The links that the user sees are not hard-coded into an HTML document,
but reconstructed from {\em Graph Query Objects} stored in an XML data
file and dynamically output to the screen (this XML file is created
using the {\tt dump} function from the {\sc Data-DumpXML} module and
is simply a dump of a complicated perl data structure into a string).
These Graph Query Objects may be created and edited by authorized
users. With this system, there is no need for a developer to create
FastGraph queries. When the need arises for a commonly desired graph,
any authorized user can create a Graph Query Object using simple HTML
forms. This process is described in more detail in the {\bf submit
GQO} section.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% ROOT GRAPHS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{ROOT Graph Page}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of ROOT Graph
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSrootgraph.eps,width=10cm}
\caption{``ROOT Graph screen shot''}
\label{SSrootgraph}
\end{figure}
The FastGraph page calls the ROOT Graph page, which displays a single
large ROOT Graph as a PNG image (shown in {\bf Figure \ref{SSrootgraph}}),
accompanied by a PostScript link (for
downloading the image in higher quality PostScript format) a Data
link (for downloading the raw data from which the graph was generated),
a ROOT file (for loading the graph directly into ROOT for further analysis)
and the SQL {\tt SELECT} statement used to obtain the data.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of Welcome page
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=VDrootflow.eps,width=12cm}
\caption{``Diagram of ROOT Graph control path''}
\label{VDrootflow}
\end{figure}
A flow-chart demonstrating the process of generating a ROOT graph is
given in {\bf Figure \ref{VDrootflow}}. ROOT is a free standing
application, which requires several pre-defined environment variables
to run. Therefore it is necessary to run a batch file in a command
shell that sets these environment variables, then calls ROOT. Since
command line arguments are passed, this requires that the arguments be
fed into the batch file, which in turn passes them to ROOT. Each step
alters the command line variables in some way, and this must be taken
into account. A preferrable method would be to pre-define ROOT's
environment variables so that the batch file were not necessary, and
then fork a process and pipe data directly into ROOT. Neither of
these are possible on a Windows server, so the elegant (and faster)
solution will have to wait for the unix version.
Currently ROOT loads one of three C++ scripts that contain the necessary
code to generate a run series, frequency distribution or scatter plot.
Although not yet implemented, the ROOT field of a Graph Query Object will
contain user-defined ROOT functions for creating arbitrary graphs.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% PLOTTING PORTAL
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Plotting Portal}
At the Plotting Portal page, the user may choose any number of data
sources, to be plotted as run series, frequency distributions or
scatter plots. The default number of graphs per page is three, but
the user may change this under {\tt NUMBER OF GRAPHS}. To select a
data source, the user chooses a table name, field name and can
optionally add a SQL constraint. Clicking on {\tt GRAPH IT} creates
multiple GD Graphs for side by side comparisons. The inability to
perform table joins is this page's only weakness.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of GD Graphs
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSgdgraphs.eps,width=12cm}
\caption{``Multiple GD-Graph screen shot''}
\label{SSgdgraphs}
\end{figure}
The Plotting Portal Results page displays three (or more) GD Graphs
tiled across the screen ({\bf Figure \ref{SSgdgraphs}}). A user could
use this to compare, for example, frequency distributions for several
different channels. The SQL command used to generate the data is
displayed below each graph, and a DATA link is provided for the raw
data. There is also a ROOT button to access the same graph with ROOT,
although this option has been turned off. The Plotting Portal will
eventually be discontinued, since FastGraph is much more powerful and
will also have multiple-graph ability in the future.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% WEB SQL
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Web SQL}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of Web SQL
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSwebsql.eps,width=12cm}
\caption{``Web SQL screen shot''}
\label{SSwebsql}
\end{figure}
Web SQL is a command line interface where the knowledgeable user can
submit raw SQL queries to the database ({\bf Figure \ref{SSwebsql}}).
Commands are entered into an
HTML form textbox, and an {\tt ENTER} button submits the command. The
SQL is then sent to the database, and the command and result are
displayed, along with a new command box. Previous commands and results
are stored on the server, so that the user can go back and edit and
re-submit commands. This is more convenient than Oracle's SQL*Plus,
which provides a command line environment, but with no ability to edit
previous commands. However, SQL*Plus does have an expanded SQL command
set, whereas the Calibration Manager is restricted to regular SQL.
Web SQL does not check the input for validity or security. It simply
passes the SQL command, along with the user's login and password, to
the database. If the user is not authorized to perform a certain
command it will be rejected by the database and an error message will
be displayed. Web SQL simply provides a user-friendly interface to
the database.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% LISTING
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Listing}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of CPS Lister
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSlisting.eps,width=12cm}
\caption{``CPS Lister screen shot''}
\label{SSlisting}
\end{figure}
The {\tt Listing} button on the welcome page calls the {\bf Script Executor}
({\bf Figure \ref{SSlisting}})
an all purpose page that runs external scripts, such as the Python scripts
{\sc CpsLister}, {\sc CpsDeleteRun}, {\sc CpsDeleteCalib},
{\sc CpsLoadDefault}, {\sc CpsLoadDummy} and {\sc CpsTreeBuilder}.
{\sc CpsLister} requires no special authorization as it
simply performs {\tt SELECT} (read-only) commands to summarize Calibration data.
The {\sc CpsLister} script is run in a command shell, and the output is simply
passed back to the web browser. The Python scripts mentioned above will be
described in more detail in a separate document, and are to be found in the
{\bf cps\_calibration} package
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% E-R Diagram
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{E-R Diagram}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Image: E-R diagram
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=ERdiagram.eps,width=12cm}
\caption{``E-R diagram''}
\label{ERdiagram}
\end{figure}
For the user's convenience, a link to an Entity-Relationship diagram
is provided ({\bf Figure \ref{ERdiagram}}). This is a diagram that
illustrates the structure of the database, displaying a summary of
each table, as well as showing how the tables are related to each
other by connecting lines. This will aid users in developing SQL
queries and creating Graph Query Objects.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SECURE LOGIN
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Clicking here causes the server to initiate a SSL (Secure Sockets
% Layer) encrypted connection with the user's browser. The effect is
% that the user will be presented with a {\em certificate}, which he or
% she must accept before continuing. After the certificate is accepted,
% a login page is loaded, which is encrypted and not visible to other
% internet users.
\subsection{Secure Login}
Whereas the Database Browsing functions listed above will work for all
users (the Calibration Manager accesses the databases using the {\tt
d0read} account by default), the Database Management functions require
the user to have an authorized database account. The user must
provide the web server with his or her password once, which is stored
locally on the server until the end of the session. To submit the
login information, he or she clicks on the {\tt SECURE DATABASE LOGIN}
button. This starts a SSL (Secure Sockets Layer) connection and loads
the login page ({\bf login.asp}) where the user enters his or her
password and the database he or she would like to use. This
(encrypted) form information is then sent to the verification page
({\bf verify.asp}), which sends a simple test query to the database
using the given password. If the query succeeds, the user's password
is stored locally as a server variable associated with the user's
session cookie. Now all database queries this user requests will be
submitted to the database with the stored password, until the session
expires and all user information is deleted.
It should be noted that for a SSL connection to be established, the web
server must present the browser with a certificate establishing its
identity. Most web businesses pay a fee to a Certificate Authority such
as VeriSign, which is trusted by most browsers, so that the user
experiences an immediate secure connection, unaware of this
acceptance of trust. We have not paid VeriSign
for such trust, but have generated our own certificate. This means that
upon ``logging in'' for the first time, the user must decide to trust our
web server, which is as simple as clicking a button on the pop-up window.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% AUTHORIZATION
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Authorization}
Once the user has submitted his or her database password, he or she is
``authenticated'' as that user as long as the session last. This
simply means that the Calibration Manager uses his password to query
the database. Authorization to perform SQL commands is handled by the
Oracle database itself, by means of ``roles''. Each user has
``roles'' determining what they may or may not do. For example, the
{\tt d0read} account may read anything, and alter nothing.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of Insufficient Authorization
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSinsufficient.eps,width=12cm}
\caption{``Authorization screen shot''}
\label{SSinsufficient}
\end{figure}
The screen capture in {\bf Figure \ref{SSinsufficient}} shows what
happens when the user {\tt herrj} attempts to delete some calibration
runs from the database. He or she is not authorized to do this per
the Oracle database's {\sc roles} (which is another topic altogether).
The database returns an error message, which the Calibration Manager
passes on to the user.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% LOGGING
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Logging}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Visio Diagram of Database Query Logging
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=VDlogging.eps,width=12cm}
\caption{``Logging of Database Queries''}
\label{VDlogging}
\end{figure}
Anonymous users are only allowed to view the database, and no record
is needed of their many {\tt SELECT} commands. But authorized users
can use the Calibration Manager to modify the database, whether this
means running the {\bf CpsDeleteRun} script or inserting and deleting
records using raw SQL commands in the {\bf Web SQL} environment. Any
such modifications should be logged, so that changes may be
reconstructed later by interested parties. Web server logs keep track
of page hits, but not input information for form variables. For these
reasons the Calibration Manager has its own logging capability,
recording any activity that might modify the database. Any query
(other than {\tt SELECT} queries) that passes through the
\verb_DBquery()_ function will be logged, and any time a script such
as {\bf CpsDeleteRun} is called, the name of the script and any error
messages generated by it will be logged as well. The log files are
stored in the directory {\tt D:\bs logs\bs CalibMgr\bs }, and each log
file is an XML file that contains one month's records. Other
authorized users can view these logs using the Log Viewer ({\tt
logbook.asp}), and may search by date, user, keyword and other factors
to reconstruct modifications. See {\bf Figure \ref{SSlogview}} for a
screen shot of this page. Currently {\sc Perl regular expressions}
are used to do searches. For example, the default value for most
fields is \verb_.*_, which is a regular expression meaning ``match any
and all possibilities''. Not everyone is familiar with regular
expressions, so eventually a more user friendly system should be
implemented.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Screen Shot of Log Viewer
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=SSlogview.eps,width=12cm}
\caption{``Log Viewer screen shot''}
\label{SSlogview}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% GRAPH QUERY OBJECTS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\bigskip
\section{Graph Query Objects}
As discussed earlier in the FastGraph section, certain types of
queries and graphs are desired quite frequently by the users. For
example, a popular graph is a simple run series of
pedestals/mips/gains (values or errors) for a particular channel in
either the FPS or CPS detector.
\subsection{Components of a FastGraph Query}
The FastGraph page was developed to make access to graphs simple and
fast for users. The notion of Graph Query Objects was conceived to
enable users to create their own FastGraph queries without
intervention of a developer, and with a modicum of effort. {\em Note
that in order to submit Graph Query Objects, the user must have a
valid Fermilab database account and use it to log in to the
Calibration Manager. }
The components necessary to construct a meaningful graph and deliver
it to the user are {\bf data} via a {\bf SQL query}, a {\bf HTML form}
for the user to choose settings, and a {\bf ROOT C++ macro} to produce
a graph. These three components (as well as some additional settings)
define a Graph Query Object, and are stored in an XML file {\tt
GQObjects.xml}. They can be submitted, edited or deleted by the user
by means of a HTML form as shown in {\bf Figure \ref{SSsubmitGQO}}.
This screen shot shows the three main components mentioned, and
farther down the screen the user will find additional settings. It
should be noted that user-defined ROOT macros have not yet been
implemented.
\begin{itemize}
\item {\bf SQL query template} \\
To get data, a SQL query is needed. First, the user can utilize {\bf
WebSQL} to edit a particular query, and once perfected, this can be
associated with a FastGraph link. For example, to get a 2-column list
containing run numbers and pedestal values for channel number 1139210
(the database uses an internal ID in which the crate, slot, HDI, SVX
and channel are encoded), sorted by run number, he or she would enter:
\bigskip
% \begin{minipage}{10cm}
\verb+SELECT CPS_CALIB_RUNS.RUN_NUMBER, CPS_pedestals.value+ \\
\verb+FROM CPS_pedestals, CPS_CALIB_RUNS+ \\
\verb+WHERE CPS_pedestals.CCR_RUN_ID = CPS_CALIB_RUNS.RUN_ID+ \\
\verb+AND CPS_pedestals.CA_ID = 1139210+ \\
\verb+ORDER BY CPS_CALIB_RUNS.RUN_NUMBER+
% \end{minipage}
\bigskip
Note that this query would also work if {\tt FPS} replaced {\tt CPS},
and if {\tt pedestals} were instead changed to {\tt mips} or {\tt
gains} (and SQL is not case-sensitive). For a FastGraph link, it
would be nice to be able to choose between FPS/CPS,
pedestals/mips/gains, value/err and also the channel ID.
Therefore, instead of simply storing a static SQL command in our Graph
Query Object, we store a {\bf SQL template}, which consists of a SQL
command with embedded ``variables''. Building on our example, when
submitting his Graph Query Object, the user would replace {\tt CPS},
{\tt pedestals}, {\tt value} and {\tt 1139210} with the variables {\em
subdet}, {\em PMG}, {\em val} and {\em channel}, so that the values
for these variables may be chosen at the time of graphing. The SQL
template he or she enters looks like this:
\bigskip
% \begin{minipage}{10cm}
\verb+SELECT {subdet}_CALIB_RUNS.RUN_NUMBER, {subdet}_{PMG}.{val}+ \\
\verb+FROM {subdet}_{PMG}, {subdet}_CALIB_RUNS+ \\
\verb+WHERE {subdet}_{PMG}.CCR_RUN_ID = {subdet}_CALIB_RUNS.RUN_ID+ \\
\verb+AND {subdet}_{PMG}.CA_ID = {channel}+ \\
\verb+AND {constraint}+ \\
\verb+ORDER BY {subdet}_CALIB_RUNS.RUN_NUMBER+
% \end{minipage}
\bigskip
Note that variable names must be surrounded by curly braces.
\item {\bf HTML form} \\
The user is also able to specify the layout of the HTML form that will
be used to select between {\tt CPS} and {\tt FPS}, etc. Instead of
requiring the user to be familiar with the (simple but always
annoying) syntax of HTML forms, it is only necessary for the user to
indicate with a couple HTML tags how they want the form layed out.
For example, an entry corresponding to our example would be:
\bigskip
\verb+{val} of {pmg}
+ \\
\verb+on channel {channel}
+ \\
\verb+constraint: {constraint}+
\bigskip
Now, on the FastGraph page, the user will see either pulldown menus or
textboxes in which they will choose e.g. {\tt value}, {\tt pedestals},
the channel number, and any additional constraints, such as an
additional {\tt WHERE} clause.
Note, however, that the variable {\em subdet} is missing from the HTML
form! This particular variable name is a special reserved variable.
On the welcome page, the user chooses which subdetector he or she is
interested in, and this setting is automatically inserted into the
{\em subdet} part of the SQL template.
It should also be noted that {\tt channel} and {\tt multichannel} are
special variable names. {\tt channel} may be entered by the user as
an internal database ID (e.g. {\tt 1139210}) or in a more user
friendly format (e.g. {\tt 0x51/10/6/2/10}). {\tt multichannel} is
similar but allows the use of wildcards. This is explained in more
detail on the Calibration Manager Help page.
\item {\bf ROOT macro} \\
Once the choices have been made and the SQL query has resulted in actual data,
this data must be passed to ROOT, which will use a ROOT C++ macro to create
a graph. In future versions, the user will be able to enter in a ROOT macro
in the same web form (there is already a space for it), but this feature has
not yet been implemented.
Instead, there are four ROOT macros written by the original developer and
currently available for use, all contained in the directory \\
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs }. These macros are described here:
\begin{itemize}
\item {\bf Run Series } \\
This macro is contained in the file {\tt runseries.C}. It takes as
input two columns of data and produces a run series plot where the
first column is plotted along the x-axis and the second along the
y-axis (the SQL command that gets this data should sort it by the
first column with the {\tt ORDER BY} command).
\item {\bf Scatter Plot} \\
This macro is contained in the file {\tt scatter.C}.
It takes as input two columns of data, just like the Run Series macro,
but the data need not be sorted, as a scatter plot consists of points with no
connecting lines.
\item {\bf Histogram} \\
This macro is contained in the file {\tt freqdist.C}.
It takes as input one column of data and plots a frequency distribution
histogram of this data.
\item {\bf Profile Plot} \\
This macro is contained in the file {\tt profile.C}.
It takes as input two columns of data, where there may be many repetitions
in the first column. It then uses the {\sc TProfile} object to make a
profile plot. For more information on what a profile plot is, consult the ROOT
documentation available for download at \verb+http://root.cern.ch/+.
\end{itemize}
\end{itemize}
\begin{figure}[ht]
\centering
\epsfig{figure=SSsubmitGQO.eps,width=12cm}
\caption{``Submit Graph Query Object screen shot''}
\label{SSsubmitGQO}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SERVER
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\bigskip
\section{The Server}
The Calibration Manager is currently installed on a Dell Optiplex
GX240 with 1.5GHz clock speed and 256 MB RAM, and one 20 GB hard disk.
So far, the hard disk has two NTFS partitions, a 4 GB partition ({\tt
C:}) for the {\sc Windows 2000 Advanced Server} operating system and
software, and a 2 GB partition ({\tt D:}) for the Calibration Manager
(this is for security reasons, to prevent crackers from ascending the
directory tree to access system files). This leaves 14 GB, part of
which has been partitioned as Linux {\tt ext2} file systems. It is a
dual-boot machine, in anticipation of a future transition to Linux
Apache.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SOFTWARE
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\bigskip
\section{The Software}
The Calibration Manager consists of a set of {\sc Active Server Pages}
(ASP) running with an {\sc Internet Information Service (IIS) 5.0}
Server on a {\sc Windows 2000} machine. ASP pages fulfill the same
role as CGI scripts, in that they enable the server to generate
dynamic web content. ASP allows a developer to embed script code
inside an HTML document, as with PHP, with the added convenience that
many different scripting languages may be used (whereas PHP requires
one to learn a new language). For the Calibration Manager, {\sc
PerlScript} has been used as the scripting language, which is
identical to the Perl language in all but one or two minor details
(for unknown reasons, the {\tt system} command is not available in
{\sc PerlScript}, but this can be circumvented by using an ASP
extension called {\sc ASPexec}. We have purposely avoided using the
PHP-like features of ASP (embedding multiple snippets of code inside
an HTML document), which means that all these pages consist entirely
of {\sc Perl} code and may be easily converted into {\sc Perl} CGI's
(or {\sc mod\_perl}) and run on an Apache server (either on a Windows
or Linux machine), if that is desired.
Currently the Calibration Manager uses two very different graphing
packages, GDGraph and {\sc ROOT}. {\sc GDGraph} is a {\sc Perl}
module using the {\sc GD} graphing functions to create basic graphs,
and has the advantages of ease of use (simple graphs can be easily
designed within the {\sc Perl} environment) and speed (Perl is already
loaded and running). {\sc ROOT} is an extremely powerful and flexible
set of C++ libraries that can perform many graphing functions as well
as data analysis. However, producing even the simplest graph involves
writing a nontrivial C++ program, which runs in its own shell. The
Calibration Manager accesses {\sc ROOT} by spawning an external
command shell with the {\sc ASPexec} ASP extension, and passing data
to it with a temporary file. Graphs are created by pre-defined C++
``scripts'' loaded into ROOT. There are currently 4 C++ scripts
written by the original developer for {\bf Run} {\bf Series}, {\bf
Scatter}, {\bf Histogram} and {\bf Profile} plots, and a planned
feature is for users to create and submit their own C++ scripts for
data analysis.
ROOT outputs graphs in {\sc PostScript} format, which is unsuitable
for easy viewing in a web browser. The {\sc ASPexec} extension is
used again to call {\sc GhostScript} and {\sc ImageMagick} to convert
the {\sc PostScript} file into the widely supported {\sc PNG} image
format, which is displayed in the user's browser.
The server on which the Calibration Manager runs has {\sc Oracle
Client} software installed, and the ASP pages send SQL commands to the
Fermilab database using the ASP interface to Microsoft's {\sc ActiveX
Data Objects} (ADO), which communicates with the {\sc Oracle client}
and then with the data server at Fermilab. Generally, all data
retrieved from the database is stored in a temporary file and kept for
a few hours so that the user can keep looking at it.
{\bf Figure \ref{VDdatalayers}} demonstrates the layers through which
the data travels.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Visio Diagram illustrating Data Layers goes here
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=VDdatalayers.eps,width=12cm}
\caption{``Data Layer Diagram''}
\label{VDdatalayers}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SECURITY
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\hrule
\bigskip
\section{Security}
\subsection{Use of Cookies}
HTTP servers do not maintain ``state'', that is, the server receives a
request for a web page and sends a file back to the user without being
aware of the fact that the user is sitting down to a session and
requesting pages in a logical sequence. Netscape invented the {\em
cookie} to surmount this shortcoming, which is a simple tag that the
server and client send back and forth to identify the user and keep
track of 'state' information. In its current implementation, the
Calibration Manager runs on a Microsoft {\sc IIS} 5.0 web server,
which is configured by default to present the user with a ``{\sc
session}'' cookie. This cookie has only one {\tt name=value} pair
that contains a unique encrypted session number generated for that
user. Thus, when a user visits any page on the site, he will receive
the session cookie, and after 20 minutes of inactivity, the cookie
expires. Microsoft built this feature in for the webmaster's
convenience, so he or she would not have to create his own cookie.
For anonymous users, the Calibration Manager makes no use of this
built-in cookie whatsoever. It ``remembers'' which subdetector and
database are selected by passing this information from one page to the
next as hidden variables, which are embedded in the HTML document.
Hidden variables are fully visible to anyone, insecure and can be
tampered with by the user, but we do not pass any sensitive data in
this way, so there is no problem.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Visio Diagram illustrating Security Setup
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[ht]
\centering
\epsfig{figure=VDsecurity.eps,width=12cm}
\caption{``Security Overview''}
\label{VDsecurity}
\end{figure}
For authorized users, the Calibration Manager would like to keep track
of their database passwords while the session continues, to avoid
encrypting all communication with SSL (this slows performance).
Transmitting them as hidden variables would be highly insecure, so it
now uses the session cookie to identify the user, and stores the
password locally on the server. It gets the password in the first
place by setting up an encrypted SSL session for the {\bf login.asp}
page, then ends the session and uses the cookie to match the user with
his password. {\bf Figure \ref{VDsecurity}} demonstrates the link
that is secured via SSL for acquiring the user's password. The link
between the Calibration Manager's server and the Fermilab database is
probably unencrypted, but since the planned physical location for this
server is on the Fermilab network, sniffers on the internet will not
be able to see any of the data transferred. If the Calibration
Manager is ported to unix, it will be able to run locally on a
Fermilab server and issues of data security between the Calibration
Manager and the database server will be nonexistent.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% MAINTENANCE
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\bigskip
\section{Maintenance}
There are currently two Perl scripts for performing maintenance on the
Calibration Manager: {\bf cleanup.pl} and {\bf backup.pl}, both stored
in the {\bf maintenance\bs } directory.
\subsection{Cleanup}
As is the case with CGI's, ASP applications run over the web and
therefore do not naturally preserve state. But the Calibration
Manager enables users to make queries and get graphs, keeping track of
all their requested data for a short duration. Graphs and data are
all stored in temporary files on the server, which without any
maintenance would build up quickly. So a separate Perl script {\tt
cleanup.pl} has been written to delete any temporary files more than 6
hours old. It simply stats each file in temporary directories to get
the time of last modification. This script is run every hour of the
day by the server, as a Windows {\em Scheduled Task} (on a linux
server the same script would be run as a cron job), and is run from
the directory in which it resides.
\subsection{Backup}
While the code is being developed, it is a good idea to make regular
backups. Currently all the Perl and Python scripts fit on one floppy
disk, and the Perl script {\bf backup.pl} puts them there. Although
{\bf backup.pl} is stored on the server, it is meant to be run from
the root directory of a floppy disk. It copies all files
(recursively) from the following directories onto the floppy: \\
\begin{itemize}
\item
{\tt D:\bs www\bs cgi-bin\bs }
\item
{\tt D:\bs www\bs doc\bs }
\item
{\tt D:\bs www\bs maintenance\bs }
\item
{\tt D:\bs www\bs script\bs }
\item
{\tt D:\bs www\bs secure\bs }
\item
{\tt D:\bs www\bs style\bs }
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% APPENDIX
%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% SOFTWARE COMPONENTS - INSTALLATION CHECKLIST
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{APPENDIX}
\subsection{Software Components to be Installed}
The {\tt C:} partition contains the {\sc Windows 2000 Advanced Server}
Operating System as well as various applications. Here is a list of
these applications, which can also be found in the file {\tt
INSTALL-W2K}. This list may be used as an installation checklist, as
it contains all components necessary for running the Calibration
Manager.
Note that for each entry, the second line contains the directory name
where the software lives on the Calibration Manager server, the next
line (optional) contains a link for downloading the software, and the
next line (optional) contains additional notes.
\begin{itemize}
\item
{\bf Windows 2000 Advanced Server, Service Pack 2, and all critical updates} \\
{\tt c:\bs WINNT\bs } \\
remember to check Critical Updates daily
\item
{\bf ActiveState Perl 5.6.1 Build 631} {\sc free} \\
{\tt c:\bs Perl\bs } \\
{\tt http://www.activestate.com}
\item
{\bf Perl modules} {\sc free} \\
{\sc GDGraph, GDGraph3D, Win32-ASP, Data-DumpXML, Array-RefElem} \\
(obtain these with {\sc PPM} that comes with ActivePerl, using commands {\tt SEARCH, INSTALL})
\item
{\bf Python 2.2} {\sc free} \\
{\tt c:\bs Python22\bs } \\
{\tt http://www.python.org/2.2/} \\
{\sc DCOracle2} has been tested successfully with this release of Python.
This is NOT ActiveState Python, which also has a version 2.2, and which
does not work with {\sc DCOracle2}.
\item
{\bf Python package} {\sc free} \\
{\sc DCOracle2} \\
{\tt c:\bs Python22\bs DCOracle2\bs } \\
The Python package for connecting to an Oracle database
\item
{\bf WinZip} {\sc free} \\
{\tt c:\bs Program Files\bs WinZip\bs } \\
{\tt http://www.winzip.com/} \\
General Windows compression utility
\item
{\bf AFPL GhostScript 7.04} {\sc free} \\
{\tt c:\bs Program Files\bs gs\bs } \\
\verb+http://www.cs.wisc.edu/~ghost/doc/AFPL/+ \\
Used for converting {\sc PS} $\rightarrow$ {\sc PNG}
\item
{\bf ImageMagick 5.4.2-3} {\sc free} \\
{\tt c:\bs Program Files\bs ImageMagick\bs } \\
{\tt http://brie.bchem.washington.edu/man/ImageMagick/ImageMagick.html} \\
used for rotating {\sc PNG} images 90 degrees
\item
{\bf ROOT Production version 3.02/07 with Install Shield} {\sc free} \\
{\tt c:\bs Program Files\bs root\bs } \\
{\tt http://root.cern.ch/} \\
I have never gotten the Install Shield to work for uninstalling {\sc ROOT}.
\item
{\bf Oracle Client} \\
{\tt c:\bs ORANT\bs } \\
This is an {\sc Oracle} Client written for {\sc NT}. You must do the custom
installation and select all files, and install to the default
directory {\tt c:\bs ORANT}. Installing to other directories will
likely result in failure.
There is no un-install option. I tried removing
it once, and had to manually edit the registry (and restart several times)
to get rid of it.
Also, don't forget the crucial file {\tt tnsnames.ora} that goes in
{\tt c:\bs ORANT\bs NET80\bs ADMIN\bs },
or you will curse the day of your birth.
\item
{\bf ASP objects} {\sc free} \\
{\sc ASPexec} \\
{\tt c:\bs Program Files\bs ASPexec\bs } \\
{\tt http://www.serverobjects.com/products.htm\#free} \\
To get this working, you must register the DLL in the directory \\
{\tt c:\bs WINNT\bs system32\bs }
with the command {\tt regsvr32 aspexec.dll}
\item
{\bf Norton Corporate Antivirus} \\
{\tt http://www.physics.lsa.umich.edu/phys-manage/} \\
A virus detection utility obtainable from the Physics Department
\item
{\bf SFTP} {\sc free} \\
{\tt c:\bs Program Files\bs SSH Secure Shell\bs } \\
{\tt http://www.ssh.com/} \\
a very user-friendly secure FTP client for {\sc Windows},
that supports drag-drop and persistent connections.
\item
{\bf putty} {\sc free} \\
{\tt c:\bs WINNT\bs system32\bs } \\
{\verb+http://www.chiark.greenend.org.uk/~sgtatham/putty/+} \\
A SSH client for {\sc Windows} \\
(I put it in {\tt c:\bs WINNT\bs system32}
so I can go to \\
{\tt Start $\rightarrow$ Run $\rightarrow$ 'putty'}).
\item
{\bf emacs} {\sc free} \\
{\tt c:\bs Program Files\bs emacs-21.2\bs } \\
The best editor in the world. Especially for {\sc Perl}
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% FILE LIST
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\hrule
\bigskip
\subsection{File List}
The {\tt D:} partition (2 GigaByte NTFS partition) contains the web virtual directory.
The base of the web virtual directory, where files get served, is {\tt D:\bs www\bs }.
The Calibration Manager consists of {\sc ASP} pages living in this directory,
and a list of all the important files can be found below.
Directories are listed before files, and otherwise it is in alphabetical order
(This is the exact order they are listed in the Windows 2000 File Explorer,
when you click on the {\tt Name} tab).
\begin{itemize}
\item
{\tt D:\bs logs\bs CalibMgr\bs } \\
XML logs created by the Calibration Manager to indicate modifications
made to the database, as well as timing logs for performance monitoring.
\item
{\tt D:\bs temp\bs } \\
temp directory used for storing intermediate image files
\item
{\tt D:\bs www\bs cgi-bin\bs } \\
The bulk of the Calibration Manager lives here.
\item
{\tt D:\bs www\bs cgi-bin\bs charts\bs } \\
a directory for storing temporary files \\
(cleaned out periodically by {\bf cleanup.pl}).
\item
{\tt D:\bs www\bs cgi-bin\bs commands\bs } \\
a directory for storing temporary files \\
(cleaned out periodically by {\bf cleanup.pl}).
\item
{\tt D:\bs www\bs cgi-bin\bs data\bs } \\
These are temporary directories for charts/graphs, Web SQL command
histories, and SQL request data \\
(cleaned out periodically by {\bf cleanup.pl}).
\item
{\tt D:\bs www\bs cgi-bin\bs GQO\bs GQObjects.xml} \\
Directory for storage of {\em Graph Query Objects}.
\item
{\tt D:\bs www\bs cgi-bin\bs GQO\bs GQObjects.xml} \\
The user-defined {\em Graph Query Objects} are stored in this file.
\item
{\tt D:\bs www\bs cgi-bin\bs graphs\bs } \\
a directory for storing temporary files \\
(cleaned out periodically by {\bf cleanup.pl}).
\item
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs freqdist.C},\\
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs profile.C},\\
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs runseries.C},\\
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs scatter.C}\\
C++ ROOT scripts for generating graphs.
\item
{\tt D:\bs www\bs cgi-bin\bs ROOT\bs runroot.bat} \\
A DOS batch file that sets the necessary environment variables and
then runs root with the appropriate C++ script.
\item
{\tt D:\bs www\bs cgi-bin\bs constants.inc} \\
Perl include file containing many constants, used by all the ASP pages.
\item
{\tt D:\bs www\bs cgi-bin\bs editGQO.asp} \\
Edit Graph Query Objects (edit, copy, delete).
\item
{\tt D:\bs www\bs cgi-bin\bs fastgraph.asp} \\
The FastGraph page - ROOT graphs are one click away
\item
{\tt D:\bs www\bs cgi-bin\bs functions.inc} \\
Perl include file containing commonly used functions for HTML form
output, Database queries, and other useful stuff needed by all the ASP pages.
\item
{\tt D:\bs www\bs cgi-bin\bs graphs.asp} \\
The Plotting Portal Results page (multiple GD Graphs)
\item
{\tt D:\bs www\bs cgi-bin\bs portal.asp} \\
The Plotting Portal page (select your data sources here)
\item
{\tt D:\bs www\bs cgi-bin\bs rootscript.asp} \\
The Root Script page that displays ROOT graphs
\item
{\tt D:\bs www\bs cgi-bin\bs script.asp} \\
Script Executor page. Runs (currently Python) scripts of any kind.
\item
{\tt D:\bs www\bs cgi-bin\bs specialvars.asp} \\
Help page explaining special Graph Query Object variables.
In the future it will be part of a more general help page.
\item
{\tt D:\bs www\bs cgi-bin\bs submitGQO.asp} \\
Form for submitting user-defined Graph Query Objects
\item
{\tt D:\bs www\bs cgi-bin\bs websql.asp} \\
Command-line SQL environment (with command history).
\item
{\tt D:\bs www\bs cgi-bin\bs welcome.asp} \\
The main welcome page
\item
{\tt D:\bs www\bs D0CM\bs welcome.asp} \\
This is a pointer to the real {\tt welcome.asp} (just in case anyone still has
the old URL we handed out at the very first video-conference presentation).
\item
{\tt D:\bs www\bs doc\bs CalibMgr.pdf} \\
This document.
\item
{\tt D:\bs www\bs doc\bs CalibMgr.proj} \\
A list of files to be analyzed by {\tt pcad}.
\item
{\tt D:\bs www\bs doc\bs CalibMgr.project} \\
A subroutine map of the entire Calibration Manager project generated with
the utility {\tt pcad} (written by Jeremy Herr). Useful for developers.
\item
{\tt D:\bs www\bs doc\bs CalibMgr.ps} \\
A PostScript version of {\tt CalibMgr.pdf}
\item
{\tt D:\bs www\bs doc\bs GQO.txt} \\
a map of the Graph Query Object's Perl data structure (for developers).
\item
{\tt D:\bs www\bs doc\bs INSTALL-W2K} \\
Installation instructions for Windows 2000 IIS/ASP.
\item
{\tt D:\bs www\bs doc\bs passed\_data.txt} \\
an abortive attempt at cataloging which hidden variables go where
(please don't refer to this; it is unfinished).
\item
{\tt D:\bs www\bs doc\bs README}
\item
{\tt D:\bs www\bs images\bs } \\
Fancy picto-buttons and such.
\item
{\tt D:\bs www\bs maintenance\bs backup.pl} \\
Back up the important files to floppy.
\item
{\tt D:\bs www\bs maintenance\bs cleanup.pl} \\
Delete old temporary files generated by WebSQL and FastGraph etc.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% From cps_calibration/script README file
% These might appear in the calibration manager someday?
%
% CpsExcludeChannels.py - adds channels to the list of excluded channels
% ('parameters') for a given collection
%
% Cft2Cps.py - copy from CFT on-line calibration database
% to CPS off-line calibration database. This is slower
% than using calDbTransfer.py but more direct.
%
% CpsChannelMap.py - fill in the channel address map from a flat file.
% (NB: you probably want something else, this was just for testing)
%
% CftOnlLister.py - list summary information about the CFT on-line
% calibration database
\item
{\tt D:\bs www\bs script\bs CpsDeleteCalib.py} \\
A Python script that deletes calibrations from the database
(not yet implemented).
\item
{\tt D:\bs www\bs script\bs CpsDeleteRun.py} \\
A Python script that deletes calibration runs from the database.
\item
{\tt D:\bs www\bs script\bs CpsLister.py} \\
A Python script that generates a summary of calibration data in the database.
\item
{\tt D:\bs www\bs script\bs CpsLoadDefault.py} \\
A Python script that loads default values (zeros for pedestals and mips,
ones for gains) into the database (not yet implemented).
\item
{\tt D:\bs www\bs script\bs CpsLoadDummy.py} \\
A Python script that loads dummy data into the database (not yet implemented).
\item
{\tt D:\bs www\bs script\bs CpsTreeBuilder.py} \\
A Python script to collect calibration runs (separately for pedestals,
mips, or gains) added to the 'bottom' of the database and
'index' them to create 'Calibrations' (not yet implemented).
\item
{\tt D:\bs www\bs secure\bs } \\
This directory is designated as an SSL directory, which means that all ASP
pages in this directory are sent over an encrypted SSL link.
\item
{\tt D:\bs www\bs secure\bs logbook.asp} \\
Page for viewing the Calibration Manager's database modification logbook.
\item
{\tt D:\bs www\bs secure\bs login.asp} \\
Page for retrieving the user's database login and password.
\item
{\tt D:\bs www\bs secure\bs logout.asp} \\
This page destroys session information about the user stored on the server.
\item
{\tt D:\bs www\bs secure\bs verify.asp} \\
This page sends the user's login and password to the database and
logs them in if the query is successful.
\item
{\tt D:\bs www\bs style\bs D0.css} (that's a zero) \\
The main Cascading Style Sheet file used for the Calibration Manager
\item
{\tt D:\bs www\bs style\bs indented.css} \\
Some of the pages look better with 5\% left and right margins
\item
{\tt D:\bs www\bs ADOPS.inc} \\
Include file containing ADO database access constants
used by {\tt D:\bs www\bs cgi-bin\bs functions.inc}
(sub {\tt DBquery}). You include it
using the {\sc SSI} command: \\
\verb++ \\
On a {\sc Windows} server, {\sc SSI} commands
are automatically enabled for {\sc ASP} pages.
\item
{\tt D:\bs www\bs index.html} \\
Redirects the user to the welcome page.
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% CODE SNIPPETS
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\hrule
\bigskip
\subsection{Code Samples}
% \begin{figure}[ht]
% \centering
% \epsfig{figure=code_sample_rootscript.eps,width=12cm}
% \caption{``Code Sample: Calling ROOT''}
% \label{code_sample_rootscript}
% \end{figure}
Several sample excerpts from the code of the Calibration Manager are
included below, for the user's perusal. These excerpts are meant to demonstrate
some of the Calibration Manager's key functionality, including database queries,
the {\sc ROOT} interface, and the general form of an ASP page.
\begin{itemize}
\item[{\tt DBquery()}]
Here is the perl code of the function {\tt DBquery()}, which is defined in the
include file {\tt functions.inc}. {\sc WebSQL}, {\sc FastGraph} and {\sc Plotting Portal}
all call this function whenever data from the database is needed. {\tt DBquery()}
simply opens a database connection using ASP and ADO objects and methods,
grabs all the data, and returns it along with any error messages.
{\scriptsize
\begin{verbatim}
sub DBquery
{
my ($dsn, $username, $password, $hostname, $comment, $mySQLrequest) = @_;
my $mySQLresult = "";
my ($rowsptr, $titlesptr);
my ($conn, $rst);
$conn = $Server->CreateObject("ADODB.Connection");
$rst = $Server->CreateObject("ADODB.Recordset");
$rst->{CacheSize} = $DBCACHE; # force ADO to get many records at once (faster)
$OLEDBstring = "Provider=MSDASQL.1;Data Source=$dsn;User Id=$username;Password=$password;";
$conn->Open($OLEDBstring); # open DB connection
$rst->Open($mySQLrequest, # get recordset
$conn) ;
my $numfields = $rst->Fields->{count};
# NOTE: if you do a bad SQL query, so that $numfields is undefined (""),
# this loop will cause the inetinfo.exe bug that requires restarting the machine
# which is why I check to make sure $numfields is ok
if(not $conn->Errors->{Count} and $numfields ne "") {
$mySQLresult = "";
&debug(5, "DBquery: no errors detected for SQL request");
# Get the field names into the array @$titlesptr
for($fnum=0; $fnum<$numfields; $fnum++) {
push(@$titlesptr, $rst->Fields($fnum)->{name});
}
# slurp the entire recordset into a 2D array (fast)
# NB: idiotically enough, GetRows indexes the array transposed from
# what a reasonable person would do, so printing the array out is
# a pain. But it's fast and efficient
$rowsptr = $rst->GetRows();
}
else {
$mySQLresult = &getDBerr($conn);
}
if ($mySQLrequest !~ m!^select!ims) {
&make_log_entry($username, $hostname, $mySQLrequest, $mySQLresult, $dsn, $comment);
}
return ($mySQLresult, $titlesptr, $rowsptr);
}
\end{verbatim}
}
\item[{\tt rootscript.asp}]
The next code sample is taken from the file {\tt rootscript.asp}, which is responsible
for creating {\sc ROOT} graphs. This sample demonstrates the necessary parts of
an ASP document. Note that the applications {\sc GhostScript} and {\sc convert}
(from ImageMagick) are called in addition to {\sc ROOT} to manipulate the image's
format (if {\sc ROOT} could output {\sc GIF} images in batch mode as well as in
user mode, these extra time-consuming manipulations would be unnecessary).
{\scriptsize
\begin{verbatim}
# Set ASP language to (Active State) PerlScript
<% @LANGUAGE = PerlScript %>
# Delimiter to mark beginning of ASP
<%
# needed for some special data structures
use Win32::OLE::Variant;
# load in huge list of constants and functions
require ("d:\\www\\cgi-bin\\constants.inc");
require ("d:\\www\\cgi-bin\\functions.inc");
# check for hidden variables
&die_gracefully() unless &status_ok();
# Get filename from constants.inc
$THIS_DOC = $rootgraphs_page;
$THIS_TITLE = "ROOT Graph";
# Get session variables from server
($AUTHLEVEL, $LOGGED_IN, $CURR_HOST,
$USER, $PASSWORD) = &get_user_info();
# Get current database and subdetector
$DB = &get_form_var("DB");
$SUBDET = &get_form_var("SUBDET");
$DSN = $DB;
# make the HTML header
&initialize_buffer();
&pr("\n");
&pr("
$SQL\n"); # Make a button to link back to the welcome page &pr("