V. Calibration Manager
The Calibration Manager is a web-based Graphical User Interface that allows any physicist anywhere with a web browser to read Preshower Calibration Databases. Authorized physicists may also use it to manage the database, making modifications and updating it.
The Calibration Manager is a useful tool for examining the behavior of the pedestals, mips or gains for any specified channel over time. It can be used to verify that the TreeBuilder logic is functioning properly and it is the portal to be used for updating the calibration data based on post-processing analyses. The Calibration Manager logs all operations on the Calibration Database and thus serves as the database logbook.
Advantages over existing database client software
Technical Components
In its current form, the Calibration Manager consists of ten or so Active Server Pages (ASP) files, living on a Microsoft Windows Internet Information Services (IIS) 5.0 web server. One benefit of ASP is that any scripting language with a “COM-compliant scripting engine” can be used. This includes many mainstream languages, and we have used PerlScript for all the pages so far, because of its power and widespread use.
The Calibration Manager connects to and queries the D0 Oracle databases at Fermilab. To do this, our server uses Oracle Client software, and the interface between ASP and the Oracle Client is provided by Microsoft's ActiveX Data Objects (ADO). This provides a high-level way for the developer to submit queries to a database. We have been able to optimize ADO for speed somewhat, and using our server in Ann Arbor, we are able to get 73 000 values per second on average from the Fermilab database (where “values” are rows times fields). It may be possible to optimize ADO further to get huge datasets faster, and we plan to move it on-site at Fermilab, we expect even better performance there.
One of the more powerful features of the Calibration Manager is the graphing capability. For fast generation of run-series, frequency distribution and scatter plots, the Perl module GD::Graph is used. Since it is native to Perl it works smoothly and quickly. For higher quality graphs with the added possibility of data analysis, ROOT is used. A Perl script in an ASP page starts ROOT in a new command shell and calls upon a C++ script that generates a graph, returning it to the user as a GIF image.
Overview
Currently the Calibration Manager consists of the following ASP files,
|
welcome.asp constants.inc functions.inc portal.asp graphs.asp fastgraph.asp rootscript.asp websql.asp logbook.asp script.asp |
The welcome page Include file – important constants Include file – commonly used functions Plotting Portal – choose data sources Plotting Portal – results FastGraph – choose a quick query ROOT access – interface with ROOT Web SQL Interface LogBook viewer Script Executor |
139 lines 245 lines 1005 lines 344 lines 1126 lines 119 lines 189 lines 585 lines 581 lines 60 lines |
which are laid out in the following manner:
FastGraph
The FastGraph page offers quick access to graphs of popular data queries, with only a couple mouse-clicks. Physicists who work with the database make certain types of requests very frequently, which might involve several lines of SQL. For example, a popular query will be to look at the pedestal values on a certain channel X, sorted by run number. The corresponding SQL command is:
|
SELECT CPS_PEDESTALS.VALUE FROM CPS_PEDESTALS, CPS_CALIB_RUNS WHERE CPS_PEDESTALS.CCR_RUN_ID = CPS_CALIB_RUNS.RUN_ID AND CPS_PEDESTALS.CA_ID = X ORDER BY CPS_CALIB_RUNS.RUN_ID |
This query “template” is entered into a file once, and thereafter the user can simply type the desired channel number in a web form, click the button and get a frequency distribution histogram of the data.
Plotting Portal
The Plotting Portal is a more comprehensive graphing page, with greater query flexibility and the ability to display multiple graphs at once. The user selects plot type, table/field name, and additional WHERE clauses, and is presented with side-by-side graphs in one page. One data set can be viewed in different ways, or several similar data sets can be compared. The plot types that are currently available are run series graphs, frequency distribution histograms and scatter plots.
Below each graph there is a quick description of the data (SQL query and number of points) as well as links to a downloadable data file, and the same data graphed with higher quality by ROOT.
Web SQL
The Web SQL Interface provides a command-line environment for entering SQL commands. The user is presented with a web form text box, where he may enter an SQL command and click the ENTER button. The results of this query are returned, and a new text box appears at the bottom of the page, for as many commands as the user enters. This has the familiar feel of a command-line environment, and has the substantial advantage over SQL*Plus that it is possible to edit and re-submit previous commands. There are also navigational aids to jump directly to previous commands, and links for downloading results as data files.
Normal users are restricted to SELECT commands, whereas an authorized user may make database modifications, which are recorded in a log file.
Script Execution
Any external script or executable that can be run on the web server can be used with the Calibration Manager. This includes ROOT, the Treebuilder and other Python scripts for summarizing data.
Logging
The goal of the Calibration Manager is to provide one-stop-shopping for all calibration tasks, including modifying the database. Any attempts to modify the database are blocked by the Manager without proper authorization, and all authorized modifications are logged. A record is made of the date, time, database, user, SQL command, and any comments the user enters. After the damage has been done, other authorized users can browse and search the Log Book to reconstruct what changes were made.
Security
Security is a highly important issue, since many users are accessing the database with varying intentions. To gain authorized access, we currently have a special account to which the user must log in before gaining authorized access. This login is protected by Secure Sockets Layer (SSL), the standard encryption used in web business transactions. For a more permanent solution, we plan to use encrypted cookies, so that a user logs in, gets a cookie, and is allowed to do the tasks appropriate to his authorization level.
Licensing
The Oracle Client software must be purchased for every machine that uses it, so installing it on every machine that might need database access would be expensive. This is avoided with the Calibration Manager, since it handles all the database access for the user.
Portability/Maintainability
Although the Calibration Manager was written using ASP/PerlScript on a Windows IIS server, we have considered from the beginning the possibility of eventually migrating it to other systems. All file and directory names are stored in the constants.asp file, and all functions containing ASP-specific code are stored in the functions.asp file and clearly labeled. If it became necessary to migrate to a (Windows or Linux) Apache server, the task would be very simple, involving rewriting the database query function and a couple other functions, and changing the first few lines of all the .asp files.
Modularity has been the key word for all the code, and so maintainability in general should be very straightforward. Instead of pages of HTML strewn about with interspersed ASP statements, the code consists of readable function names that call, for example, a subroutine that generates an HTML pulldown menu. This modularity would also make it easier to rewrite the entire thing in Python, if that ever becomes necessary.
- 1 -
Last saved by Homer A. Neal on 08/18/03 at 10:29 AM