The following report is available in PDF format for high quality printing.

Fourth Quarter FY 2002 Report - The National Fusion Collaboratory

Edited by D.P. Schissel1

1General Atomics (schissel@fusion.gat.com)

Overview

The focus this quarter was taking the capabilities demonstrated at the Sherwood and TTF fusion science meetings, bundling them in a form that can be used by a scientists, and supporting their usage for physics analysis to be presented at the 19^th IAEA Fusion Energy Conference and the 44^th Annual Meeting of the Division of Plasma Physics (American Physical Society: APS/DPP). Additionally, work this quarter focused on creating presentations and demonstrations by the Collaboratory Project at SC02 and APS/DPP 2002.

General accomplishments include:

∑ The first code placed on the FusionGrid, TRANSP, was released to scientists for physics analysis. This code joins the existing FusionGrid MDSplus data repositories.

∑ SCIRun is now being used to visualize fusion data stored in MDSplus.

∑ MDSplus documentation (http://www.mdsplus.org) was substantial upgraded due to its expanded usage on the new FusionGrid.

∑ Presentations on Collaboratory Project work were given at the NeSC Workshop on Applications and Testbeds on the Grid, the Workshop on Advanced Collaborative Environments, and to the EFDA-JET Fusion Programme.

∑ The Collaboratory Project and the magnetic fusion energy community were represented at several DOE sponsored planning meetings.

General

The science application needs and the resulting computer science requirements of the U.S. magnetic fusion energy research community were presented at several planning meetings this quarter. The MAGIC Document Workshop investigated the long-term (~10 years) needs for middleware and Grid technology. The DOE High Performance Network Planning Workshop addressed future high-speed network requirements created by progress within the present SciDAC program. The SciDAC Visualization Workshop examined the visualization requirements for the present SciDAC projects to determine if additional visualization research will be required to support such large-scale science. Throughout these meetings the work done by the Collaboratory Project combined with the requirements of the fusion community placed fusion science as one of the driving applications for defining future computer science research.

The work of the Collaboratory Project was presented at several different locations this quarter. The scientific and computer science staff at the EFDA-JET Fusion Programme was briefed on the status of the Collaboratory Project. Areas of mutual collaboration in both grid-enabled TRANSP computing and Access Grid meetings are being investigated. The computer science research of the Collaboratory Project was presented at both the NeSC Workshop on Applications an Testbeds on the Grid and at the Workshop on Advanced Collaborative Environments.

A substantial amount of work was done this quarter in preparation for demonstrations to be given at the APS/DPP meeting. Typically such demonstrations do not occur at this meeting thereby necessitating our creation of the entire infrastructure for the show floor. Since this meeting is the largest gathering of fusion scientists in the U.S., demonstrations and feedback on our progress will be extremely valuable. Additionally, a meeting of the Project Oversight Committee will take place at the meeting.

The Collaboratory took part in security discussions at the August ESCC meeting. Specifically, the needs of site-security were compared to the needs of grid-security. Collaboratory project members presented our needs (grid-security) and offered to work with the ESCC to help create a policy that allows both groups to accomplish their mission. It was decided by the ESCC to form a working group to investigate this area further and several members of the Collaboratory Project will participate in this group.

The Project web site continued to be updated with substantial information being added regarding the APS/DPP and SC02 meetings.

Security/Remote Computing

The Collaboratory Project deployed the TRANSP code to the FusionGrid for use by scientists at DIII-D and PPPL for data analysis to be presented at two major scientific meetings. Numerous software components were modified and deployed to create this capability. The GUI PreTRANSP, used for preparation of inputs and securely invoking the Grid enabled version of TRANSP, was further modified so that it could be released to scientists. This system orchestrates the creation of input data that is stored in MDSplus and the Run Management database, and starts the TRASNP run on the PPPL Linux cluster. All communication is secured utilizing the Globus toolkit. The PPPL TRANSP cluster was deployed with 8 dual processor 1.7Ghz Intel-Linux servers. Removing this code from the scientistsπ workstation requires a detailed monitoring capability that allows them to track the status of their runs. A new Grid monitoring system (http://nssrv1.gat.com:8000/fgm) was created for FusionGrid resources and its first use has been with the TRANSP code. The design of this system was derived from a working system used to monitor between shot processing at DIII-D. The entire TRANSP analysis chain was setup to post status information to the FusionGrid monitoring system.

The second phase of integrating Akenti with the Globus job-manager was accomplished this quarter. The job-manager can now permit or deny job execution based on whether the Resource Specification Language (RSL) parameters for the request comply with the values that the policy allows for the requestor. Examples of such parameters include job startup, termination, number of allowed CPUs, total execution time, maximum memory, and querying status.

A prototype of a FusionGrid reservation-based QoS for job execution was developed. This system allows the user to place an upper bound on job execution time. Testing on the fusion science application EFIT has begun and represents a first step toward the capability of utilizing the FusionGrid for between-pulse experimental data analysis (preemptive scheduling). Since this system is implemented using a technology preview release of OGSA, it also represents our first step in the exploration of this new technology.

Given the expanded usage of MDSplus in the new FusionGrid the MDSplus documentation was substantially upgraded (http://www.mdsplus.org) this quarter.

Visualization

Modifications were made to SCIRun in support of NIMROD data visualization to be presented at an upcoming fusion science meeting. These modifications included adding different visualization techniques, the animation of time dependent data, and the expansion of the MDSplus reader to allow a more flexible format for NIMROD data input. The design of a more generalized version of the MDSplus reader has been completed that will allow both experimental and theoretical data to be read. A 2D plotting package was implemented in SCIRun that allows multiple graphs in one window and can be used to view Image, Contour, Surface, and XY plots. This capability is required for quantitative comparison of theory with experiment.

The Personal Interface to the Access Grid (PIG) has been developed for both Linux and Windows based systems with the Linux version still in beta testing. A PIG is being tested by fusion scientists at MIT. Existing equipment was reconfigured to save Collaboratory funds and their local ESNET connection was reconfigured to allow multicast. Initial tests have the system working but with a large impact on the local area network. The Collaboratory Project continued to test the use of tiled display walls as a display surface for the Access Grid (Figure 1). The ANL mMural2 tiled display has been made operational with 2 camera support and full audio capabilities. The environment is based on the recently developed PIG extensions to the AG.

Figure 1 Access Grid running on tiled display (6 projector ANL mMural2 being driven by a Linux based PIG.

Testing the usage of tiled display walls in a tokamak control room was done at PPPL. VNC was used to share live NSTX data between a display wall and workstations connected with Gigabit Ethernet. Detailed studies have begun to develop designs for deployment of such systems in tokamak control rooms including the impact of network speeds. The prototype SciVis software has evolved into ElVis and has been demonstrated at PPPL for collaborative visualization over low speed wide area networks.

Appendix A: Non-Edited Reports from Individual Institutions

A.1 M. Papka for the ANL MCS, Futures Laboratory

∑ Continued work and testing on the use of tiled displays as a display surface for the Access Grid (AG). Using the Argonne built mMural2 display we have constructed an AG with 2 camera support and full audio capabilities. The environment is based on the recently developed PIG extensions to the AG, built using the Linux version of the software. In combination with this ANL has continued its tests of using VNC and DMX to do sharing of pieces of the desktop between users (Figs. A.1-1,2,3).

Figure A.1-1 Access Grid running on tiled display (6 projector ANL mMural2)
being driven by Linux based PIG

∑ The Personal Interface to the Access Grid (PIG) has developed now for both Linux and Windows based systems. The Windows version is available to the community off the AG download page and the Linux version is in beta testing now with a few key sites. Numerous improvements have been added to the Windows version based on community feedback, including the use of UCL vic for fully secure sessions, support for Firewire and USB cameras, and the support of both Windows 2000 and XP operating systems.

∑ Improved the video based remote visualization client, users now get a single video window where the resolution is dependent on the number of streams (and how they are tiled together). Users are able to specify the configuration in a text file that is read in at application start up. This improvement simplifies user start up by not requiring participants to put individual files together.

Figure A.1-2 Image of users in ANL Access Grid node using SCIRun
applications running on top of ANL developed Chromum H261 SPU

Figure A.1-3 SCIRun running the University of Utah fusion package on top of
Argonneπs video Chromium SPU allowing for remote visualization over the AG.

A.2 K. Keahey for the ANL MCS, Distributed Systems Laboratory

In the last quarter ANL accomplished the following tasks:

Developed an enforcement module implementing a comprehensive range of VO-wide job management policies as agreed upon at the PPPL meeting in the last quarter. The policies include job startup, termination, querying status, and other user job management functions implemented in GRAM, as well as VO-wide third-party management (changing job priority etc.). With the help of our LBNL colleagues we represented the policies as Akenti policies. The paper describing this design in detail will be published in the proceedings of the Grid2002 workshop. The working system will be presented at the APS and SC02 conferences.

Developed a prototype of a network services infrastructure with a reservation-based QoS interface module enabling users to run fusion applications as network services guaranteeing a certain QoS. In particular, the resource management system implemented by this infrastructure allows the user to put an upper bound on the execution time for network services in the infrastructure. In collaboration with our GA colleagues began testing capabilities of this infrastructure on real fusion applications. Since this infrastructure is implemented using a technology preview release of the Open Grid Services Architecture (OGSA), it also represents our first step in the exploration of this new technology. A demonstration of the prototype will be shown at the SC02 conference.

In collaboration with our colleagues from PPPL, we initiated discussion of firewalls for Grid computing at the August ESCC meeting. At that meeting, Von Welch of the Globus project gave a presentation on issues relating to the use of firewalls with Globus. Based on feedback from our collaborators we also updated the documents describing security practices in Globus and their relationship to firewalls.

Gave a presentation on the current state of the National Fusion Collaboratory at the NeSC Application Workshop in Glasgow in July. The talk described the requirements of the project, the problems we are encountering, and some proposed solutions.

Provided technical support, assisted with debugging and troubleshooting of Globus installation and associated technologies. This involved providing explanations and technical discussions of Globus functionality as well as providing examples, help with troubleshooting and answering questions.

Interacted with collaborators through teleconferences, discussions, etc.

A.3 D. Schissel for the General Atomics Fusion Group

General

∑ Facilitated a discussion at the ESNET Coordinating Committee meeting in August on the needs of site-security compared to the requirements of Grid-security (Schissel). A working group will be formed and composed of people from both disciplines and will include members of the PPDG Project.

∑ Presented the science application needs of the U.S. magnetic fusion community at the MAGIC Document Workshop in Chicago that investigated the long-term needs for middleware and Grid technology (Schissel).

∑ Presented the visualization requirements of the U.S. magnetic fusion community at the SciDAC Visualization Workshop in Salt Lake City that investigated the advanced visualization needs of the various SciDAC funded projects (Schissel).

∑ Presented the network requirements of the U.S. magnetic fusion community at the DOE High Performance Network Planning Workshop in Virginia (Schissel).

∑ Presented the work done by the National Fusion Collaboratory Project at the Workshop on Advanced Collaborative Environments in Scotland (Schissel).

∑ Presented the work done by the National Fusion Collaboratory Project to the EFDA-JET Programme in Oxford and explored areas of cooperation (Schissel).

∑ The project web site (http://www.fusiongrid.org) was maintained including updates for the APS/DPP and SC2002 meetings.

Security/Remote

∑ The GUI PreTRANSP, used for preparation of inputs and securely invoking the Grid enabled version of TRANSP was further modified and released to several scientists at DIII-D (Burruss). This system orchestrates the creation of input data that is stored in MDSplus and the Run Management database, and starts the TRANSP run on the PPPL linux cluster. All communication is secure utilizing the Globus toolkit (Burruss).

∑ A Grid monitoring system (http://nssrv1.gat.com:8000/fgm) was created for Fusion Grid resources; its first use has been with TRANSP. The design of this system was derived from a working system used to monitor between shot processing at DIII-D (Flanagan). This advanced monitoring capability was required before the grid-enabled TRANSP system was released for use by DIII-D scientists.

∑ Grid-enabled TRANSP was successfully used to perform physics analysis at DIII-D in support of the IAEA and APS/DPP meetings.

∑ Support prototype QoS work by Keahey for a preemptive scheduling SC02 demonstration by installing EFIT at ANL and supplying input data (Peng).

Visualization

∑ Minor updates and improvements to SCIRun were accomplished to support visualization of NIMROD data for Brennanπs invited talk at the APS/DPP meeting (Peng, Sandeson).

∑ A dramatic decrease in the time required to write NIMROD data from NERSC to the GA MDSplus server was accomplished by software tuning (Flanagan, Kruger). Presently, there is approximately 100 GB of NIMROD data stored in MDSplus.

A.4 M. Thompson for the Lawrence Berkeley National Laboratory

The second phase of integrating Akenti with the Globus job-manager was accomplished this quarter. The job-manager can now permit or deny job execution based whether the Resource Specification Language (RSL) parameters in the job request comply with the values that the policy allows for the requester. Examples of such parameters are the number of CPUs that the job may use, the total execution time that the job can use and the maximum memory that it can use. The Akenti policy engine support for system constraints was completed in order to support this functionality. A new interface to the Akenti server was written to allow conditional actions to be returned in a manner that C code could parse. The job-manager was modified to parse the returned conditional actions and to compare them with the RSL parameters.

The job-manager code for canceling or checking status of jobs was also modified to call out to Akenti for authorization decisions. This will allow an administative user to check on or cancel other user's jobs.

This work was done at ANL with support from LBNL. It is now is the process of being transferred to PPPL for eventual use on the production TRANSP machines.

A.5 M. Greenwald for the MIT Plasma Fusion Science Center

MDSPlus Documentation

The MDSplus data system is a key element of the developing fusion grid. This expansion in the user base has motivated us to spend significant effort to upgrade documentation. The MDSplus web site http://www.mdsplus.org/ has been extensively reorganized and several new documents added. These include an introductory section for new users and a beginners guide which covers basic MDSplus concepts, tree structures and syntax, data types, TDI expressions, the MDSplus API, C, Fortran and IDL interfaces, and the graphical tools (x-windows scopes and traverser, java scope). A glossary of MDSplus terms has also been added. Work has begun on the users guide with several sections completed including a description of the MDSplus datatypes (there are 38 supported) with details as well as a list of common operations for each; tree syntax, and file system details. Pages and scripts have been added to support downloads of the MDSplus software. Each download is now entered into a relational database allowing the development team to better track the distribution.

Personal Access Grid

The access grid (AG) is a continuous presence, multi-stream video conferencing application. This technology, used for large and small group collaborations across the internet, is described in depth at http://www.accessgrid.org/. We have set up and are testing a new version of the software supporting a 'personal access grid' node at the PSFC to enable experimentation with this technology and participation in access grid based remote meetings.

Hardware and software were added to an existing high end PC, and our ESnet connection was reconfigured to allow multicast traffic. The system was made operational with help from Argonne and Esnet personnel. Further work will be needed to resolve the severe impact that resulting multicast traffic has on our local area network. Turning on the AG "beacon" causes dozens of video and audio streams to be sent through the local ESnet router and onto our network. While multicast saves network bandwidth over shared wide-area network links, the packets flood the local switched infrastructure.

A.6 D. McCune for the Princeton Plasma Physics Laboratory

Grid-Enabled Remote Computation (L. Randerson, C. Ludesher, D. McCune)

The Grid-Enabled Collaboratory TRANSP service was readied for deployment (actual deployment for DIII-D users took place in early October 2002ãwith the Globus-secured service producing TRANSP results being used for the IAEA and APS plasma physics conferences, as planned).

The Globus-secured TRANSP deployment features:

firewall changes at PPPL to support "trusted client" machines at DIII-D.
approval of DOE Science Grid Certificates for recognized DIII-D TRANSP users.
deployment of 8 dual processor 1.7GHz Intel-Linux servers under control of a PBS scheduler for actual execution of runs.
setup of separate, single purpose TRANSP production account for each TRANSP user-- assuring privacy of each user's TRANSP results.
TRANSP production accounts allow users to:

submit TRANSP runs.
cancel or archive unfinished TRANSP runs.
view interim results of TRANSP runs during execution.

for security reasons, users cannot use their production accounts for any other purposes; this restriction is enforced by Globus/ Akenti. (PPPL Collaborators may have other accounts for full interactive access to PPPL Computing resources).

PPPL TRANSP experts have access to user production accounts purposes of debugging TRANSP run crashes (which are a normal part of operation in the analysis of experimental data).

TRANSP production scripts have been set up to feed data to a "fusion grid monitor" which allows users to monitor the status of their runs and view logfiles. Run script also have the option to send users email upon run completion.

As of the end of September 2002, deployment of the Collaboratory TRANSP run production system was well under way to meet its goal of usability in time for the November APS/DPP conference.

It is worth noting that PPPL internally has long supported TRANSP run production facilities for local users on the PPPL LAN, with many similar features. This local (non-Globus) capability has been merged with the wide area (Globus) capability to form a single unified production system. This has allowed PPPL to leverage extensive prior work and experience on TRANSP run production in the new expanded TRANSP run production service.

Collaborative Visualization (E. Feibush, S. Klasky)

A control-room practical VNC (Shared "X" environment) configuration based on a shared Display Wall and workstations connected with gigabit ethernet, was demonstrated on the PPPL wall with live NSTX data viewing applications, and was enthusiastically received by NSTX experimentalists. Detailed studies have commenced to develop designs for deployment of a such system in the NSTX control room, for deployment within the year. Such a deployment, although funded by the NSTX project, would represent utilization of software technology developed in part by the Fusion Collaboratory project.

Images of the control-room mock up demo were provided for the video being prepared for the SciDAC booth at SC2002.

VNC (Shared "X") is not expected to perform adequately over existing wide area networks. However, a multi-tiered internet-enabled visualization client-server system, "ElVis", evolved from the prototype SciVis software, has been successfully demonstrated at PPPL. This allows collaborative shared visualization of collections of small-to-medium sized datasets over a low speed wide area network.

The "ElVis" server supports two types of clients::

data provider clients request the posting of visualization windows on the ElVis server.
data display clients allow shared access to visualization windows; manipulations of display windows (e.g. to focus in on a particular feature of the data) carried out by one display client are seen by all display clients sharing the window.

Although the display clients are written in Java, data providers use a C-language API that is friendly to legacy visualization tools written in e.g. Fortran or IDL.

The ElVis software is on schedule to be demonstrated at the November 2002 APS and Supercomputing conferences.

A.7 A. Finkelstein for the Princeton University Computer Science Department

No report.

A.8 Sanderson for the University of Utah Center for Scientific Computing and Imaging

∑ Collaborative SCIRun (J. Simpson) - Completed the initial implementation of multi-casting schemes in SCIRun. The current implementation is too slow using image based rendering with reliable mutli-casting. Conversely, an unreliable implementation losses too many frames for usage. Currently we are looking at the other rendering schemes (Geometry and Z-Texture) available in SCIRun with other multi-casting schemes (Pretty Good and Unreliable). This will be presented at SC2002 as poster.

∑ SCIRun Fusion Package - (A. Sanderson) Released the next version which added new networks to the package that demonstrated different visualization techniques. In addition, modules which allow animation of time dependent data to be viewed, and modules for manipulating data are being exploited. The MDS reader has been expanded to allow a more flexible format for NIMROD data input. Given this work, a more generalized version of the reader to allow both experimental and theoretical data to be read is currently being developed (Q. Peng).

∑ Implemented a 2D plotting package using the Open Source PLplot package in SCIRun (A. Sanderson). The plotting allows the multiple graphs on one window, and can be used to view Image, Contour, Surface, and XY plots. The interface allows for multiple signals to to be viewed together. The goal of this project is to allow for basic 2D and 3D plotting within SCIRun. We are currently finishing this work for the APS meeting.

∑ Error Visualization Research (A. Sanderson) - Began initial research into visualizing 3D error measurements in SCIRun. Currently we are exploring the use of glyphs to display error for both scalar and vector data. Glyphs have proved to be useful since they offer geometry which represents the error space. Further they can be over loaded using color to give an added dimension.