1   Introduction

Back to top

2   Accomplishments in the Past Years

2.1  Software Development and Coordination

Over the past three years, the University of Michigan has expanded its role significantly in the development and coordination of the Atlas offline software. Steven Goldfarb was elected in June 2003 by the Muon Institute Board to be the Atlas Muon Software Coordinator. This superseded his previous position as the Muon Database Task Leader and expanded his responsibilities to overseeing the planning and coordination of all offline software development for the Muon Spectrometer. The board recently re-elected Steve to this role, for the next two years (until 2007). In this position, he serves on the Atlas Muon Management and Software Project Management Boards.

Since 2003, Steve has overseen a significant expansion in the scope and complexity of the software development. This has included the migration to GEANT4, Athena-based digitization, new event data and detector description models, calibration and alignment services, and the continued improvement and validation of reconstruction models capable of handling simulated and test beam data.

It should be noted that there is significant overlap in the development of the test beam software infrastructure with the work reported on H8 test beam analysis in Section XXX. In addition to testing and understanding the response of the detector, the test beam activities are used as an important means of validating and improving the software and our group’s work on the test beam analysis has contributed to several areas of the general muon software development.

Important milestones in the coordination of the Muon offline software over the past three years have included preparation for the Fall, 2003 LHCC Computing Manpower Review, the successful completion of large scale simulation production for ATLAS Data Challenge 2, the reconstruction of simulated bytestream data for the Tier 0 exercise, and the production of physics data sets, with the initial detector layout, including background and pile-up for the June 2005 Atlas Physics Workshop in Rome. Documentation, including a complete detailed WBS for the software, can be found at [http://cern.ch/muondoc/Software ].

Back to top

3   Ongoing Studies and Some Results

Back to top

4   Proposed Work for the Next Three Years

4.1    Michigan ATLAS Group Computing

The University of Michigan ATLAS Group is one of the largest groups in the ATLAS Collaboration and, as such, has made major contributions to the construction, commissioning and software development of the ATLAS detector as outlined in this proposal. We expect to play a leading role in calibrating and validating the performance of the ATLAS muon system during the era of LHC operations. Our ultimate goal, of course, is to participate fully in the ATLAS physics analysis with particular emphasis on W/Z/Higgs events with final-state muons.

The U.S. ATLAS Computing infrastructure is currently being modeled on a grid-linked system of tiered computing centers. The basic goal is to cover the computing needs of the U.S.-based physics community, concerning data processing and physics analysis. It has begun this process by selecting those institutes capable of assembling the largest amounts of CPU power, data storage, and computing expertise, in order to achieve the most powerful system for the available resources.

Considering the sheer volume of data to be processed for the LHC and the significant and unique requirements for computing, the achievement of this goal is indeed very important. Yet, we feel there is a critical aspect of the physics software requirements, foreseen in the overall ATLAS computing model, that is being overlooked by the U.S. strategy. That is the prompt processing of the calibration and alignment data by the detector communities. Our experience as members of the D0 and CDF collaborations, and our commitment to the validation of data and calibration of our detector, has given us cause for alarm, in this respect. We would like to bring this to your attention and present our attempts at a solution.

We have taken an initial step towards a significant computing capability by acquiring from the University of Michigan a cluster of used CPU's and disks (called "HYPNOS") that has been moved to our Physics Department. HYPNOS is described in detail in the section below. It is our intention to upgrade and replace obsolete components of HYPNOS during the three years of the upcoming DoE grant period. The details of this plan are also described below. Included in this plan is a request to fund the personnel required to operate the resulting computing facility.

Before describing our plans for this facility, we first motivate the scope and capability of required computing resources with a brief discussion of the kinds of calibration activities and physics analyses we intend to pursue in our group.

1.1.1  Motivation for Proposed Computing Facility

The Michigan ATLAS group is comprised, among others, of physicists who have worked with and analyzed the data from both the CDF and D0 detectors at Fermilab. This has naturally evolved into the interests resulting in the fabrication of the MDT chambers and the desire to ensure they function properly and yield good physics data. One of the many steps entailed in such an analysis, calibration of the detector and validation of its performance, can be performed periodically using the Z->mu-mu and W->mu-nu events generated during the normal course of LHC running. In one month's running at standard luminosity some 100 Million such events will be recorded. It will require 160 CPUs and 180 TB of storage to analyze this one month’s sample of events in one month’s time.

This calibration and validation is just one step in any full scale physics analysis involving the ATLAS MDT system. Work has already begun on a number of physics analyses.

The Michigan ATLAS group will play a leading role in the analysis of two-muon final states and Lambda-b polarization studies, both of which will rely on the best quality possible extraction of measurable quantities from the MDT system. This extraction will itself be extremely demanding of computational resources in the time leading up to, and extending beyond, the onset of data collection by ATLAS.

This co-existence of necessary and ongoing detector monitoring and physics analysis requires a much larger computational and storage facility than that exemplified above. Further, such a system will require ongoing attention in and of itself, both keeping the hardware functioning and maintaining an up to date and secure software environment. One-half FTE of manpower will be dedicated to these tasks, supervising hourly help for low level work and coordinating between the analysis personnel and the ATLAS software management staff to ensure a smoothly functioning system with minimal disruptions.

Some of this computational load can in principle be served by the Tier-1 and Tier-2 US-ATLAS centers. However it is unclear how such allocations will take place as no protocol has yet been defined for doing so, and there will also be stiff competition for these resources from other research groups in the US. While grid computing promises to deliver resources to the whole collaboration based upon prioritized policies, this type of capability is far from being realized and may not be effective until well after LHC startup. The only way we can be certain that we can meet our responsibilities is through use of our own, dedicated and local resources.

1.1.2  Description of HYPNOS Cluster and Proposed Computing Facility

The Michigan ATLAS group has been able to acquire a significant computing resource through our involvement with MGRID and the Center for Advanced Computing (CAC) at Michigan. The Hypnos computing cluster, formerly an NPACI resource, has been transferred to the Physics department into an appropriately modified laboratory space (2268 West Hall). The Hypnos cluster provides 128 nodes of dual AMD Athlon MP 2000 (and 2400) processors, 2 GB of RAM per node, two 100 GB hard disks per node, a 1 TB RAID array and a FastEthernet switch for interconnections. While this cluster is over two and a half years old it is still a powerful computing resource capable of running ATLAS simulation, reconstruction and analysis jobs.

Because of the age of the Hypnos nodes and the ever increasing demand for computational power and storage required to meet our ATLAS computing needs we are planning to acquire and deploy equipment in constant dollars per year. This model provides newer, more capable equipment each year, both to meet expanding demands as well as to replace outdated or failing equipment. By spreading out the purchases we are able to reap the benefits implicit in Moore’s law increases in capability for constant dollar amounts.

The following table shows the cost of providing 16 dual processor nodes per year (current equivalent is dual 3.6 GHz Xeon, 1U rack mount, 4 GB RAM), 2 "storage" nodes (disk servers, 6 TB / node in year 1) as well as funds for parts (materials and supplies).

The following table lists the capability we will acquire as a function of year.  Note for storage we list the amount acquired each year in parenthesis after the integrated total.

This plan delivers the basic required resources for Michigan ATLAS computing. 

1.1.3  Grid and Network Research and Development at Michigan

Michigan has played a significant role in networking and grids for ATLAS. Shawn McKee is the US ATLAS networking manager, co-Chairs the Internet2 High-Energy and Nuclear Physics (HENP) Working Group and is a founding co-Chair of a new Open Science Grid Networking Technical Group. For the past three years he has been the technical lead on the NSF Middleware Initiative (NMI) testbed at Michigan. He is also part of the Michigan MGRID project which is developing grid middleware and techniques easy access for distributed grid resources. Perhaps the most impact for ATLAS in the grid and network area will come from three newly funded projects on which McKee is Co-PI:

�   UltraLight: A $2 Million NSF ITR funded by MPS (Mathematical and Physics Sciences) which is exploring advanced networking infrastructure in support of LHC scale physics. In addition to a number of CMS collaborators, Michigan, Brookhaven and SLAC are participants. During the next year we plan to start integrating new capabilities into the computing models of ATLAS and CMS from this project.

�   TeraPaths: A DoE/MICS funded project at BNL, with participation of Michigan, concentrating on developing MPLS/QoS capabilities for the Tier 1, Tier 2 and eventually Tier 3 computing centers for LHC. This work is being directly integrated into the Tier 1 efforts and we are closely coupled to ESNet and other HEP sites.

�   GridNFS: A $1.2 Million NSF NMI Development project to create a “grid” aware version of NFS (Network File System) based upon the newly developed NFS V4 standard. We have an agreement to test our project within Grid3 and plan to work closely with the OSG Storage working group and CERN to make GridNFS accessible to LHC projects.

These types of projects require resources upon which to test deployments. To be most effective the resources involved should be of similar scale to the primary targeted resources. While we are in contact with the US ATLAS Tier 1 and Tier 2 sites and have some agreement to work together on testing and deployment we will still require local resources on which to make the initial developments. Having at least 8 dual processors machines, excellent network connections and at least 2 storage nodes will be critical to the successful integration of these projects into the ATLAS computing model.

4.2  Software Development and Coordination

The focus of the ATLAS Muon software development is already beginning to shift from major core infrastructure implementation to that of calibration, alignment, and detector commissioning, as we head toward the first beam of June 2007. With our group’s vast experience in the construction, testing, commissioning and analysis of the detector response, we are poised to take on the leading role in this development.

In addition to Steven Goldfarb’s position as the Muon Software coordinator, Bing Zhou’s position as the Muon Detector Validation coordinator, and the group’s increasingly focused commitment to detector calibration, Dan Levin has been asked to co-convene the Combined Muon Working Group. This position, described in Section XXX, is vital to the validation of the software for physics. His contribution will contribute important synergy to the group’s preparation to understand the detector, its data, and, ultimately, the physics it will reveal, as early as possible.

In preparation for data taking, there are a number of important software development milestones on the horizon the next few years. Immediately following the ATLAS Physics Workshop of June 2005, which will showcase the robustness and precision of the core software infrastructure, cosmic tests will be made, using MDT chambers installed in the ATLAS pit. For the software, this effort will test new calibration and alignment services, as well as the ability of the reconstruction software to correctly interpret and apply conditions data corrections. Software developed for the analysis of this data will play an important role in the Computing Service Commissioning program (aka Data Challenge 3) to begin winter 2005, and our group’s contributions to detector calibration will be vital.

By Spring 2006, all emphasis will turn toward commissioning of the installed detector, using cosmic rays and, in 2007, beam halo, which is critical for testing and understanding the MDT endcap region. Our group has a vested interest in the endcaps and is already preparing to lead these studies.

Back to top