First Quarter FY 2004 Report - The National Fusion Collaboratory

Edited by D.P. Schissel¹

¹General Atomics (schissel@fusion.gat.com)

Overview

The main focus this quarter was the collaborative control room demonstration at the SC2003 Meeting and the presentation on this concept to the ITER Grid workshop held in conjunction with the ITER Ministerial meeting.

General accomplishments include:

• A successful demonstration of the collaborative control room at the SC2003 Meeting connecting the show floor in Phoenix with the DIII-D tokamak in San Diego.
• A presentation on the collaborative control room concept was given at the ITER Grid workshop and was so well received that it is being discussed as the model for organizing the worldwide scientific collaborations required to make ITER a success.
• The DIII-D Research Opportunities Forum was broadcast using newly purchased AG technology to a worldwide audience. Several presentations were given remotely.
• The User Oversight Committee met in conjunction with the 45th APS/DPP meeting to review the NFC project.
• Began to formulate a work plan on the next several years worth of work for the NFC project.

General

The collaborative control room concept was successfully demonstrated at the SC2003 Meeting where the show floor in Phoenix was connected to the DIII-D tokamak in San Diego. This concept and demonstration unifies the previous work of the NFC project into a single concept to support experimental fusion research. To be fully functional, the collaborative control room requires (1) secured computational services that can be scheduled as required, (2) the ability to rapidly compare experimental data with simulation results, (3) a means to easily share individual results with the group by moving application windows to a shared display, and (4) the ability for remote scientists to be fully engaged in experimental operations through shared audio, video, and applications. The SC2003 demonstration showed all of these pieces working together in a mock up of DIII-D operations in San Diego being supported by off-site collaborators located in Phoenix.

AG technology allowed for shared audio and video as well as shared applications. The offsite collaborators could hear DIII-D announcements and see via a web interface the state of the pulse cycle, the status of data acquisition, and the state of between pulse data analysis. The fusion visualization application ReviewPlus was shared between the two sites allowing for joint scientific exploration. Between pulse data analysis of the plasma shape (EFIT running at PPPL) was conducted on FusionGrid through a computational reservation system that guaranteed a specific analysis to be completed within a set time window. Additionally, the TRANSP computational service was run at PPPL for the first time between pulses giving scientists data that was previously only available after the experimental day had ended. The offsite team was able to collaborate more efficiently amongst themselves by being able to share their personal display with the room's shared display. This capability allowed visualizations to be efficiently compared side-to-side for debate before reporting results back to the DIII-D control room.

Although the demonstration was successful, it clearly highlighted areas needing additional work. Those areas that need more time than the remaining six months on the present grant will be included in the follow-on proposal

The NFC Project attended the 45th APS/DPP meeting where a booth with posters and handouts were placed near the main scientific poster room. This allowed the NFC project to detail accomplishments and how fusion scientists can start to utilize these new capabilities. It also allowed the NFC project to receive direct feedback from the user community. The NFC Project's User Oversight Committee held a meeting in conjunction with the APS/DPP meeting. Although praise was given to our work, the committee highlighted areas where either more emphasis or less emphasis should be placed. Details will be made available in a written report that can be used to adjust our work plan.

Based on the collaborative control room demonstration at SC2003, the NFC Project was asked to make a presentation at the ITER Grid Workshop held in late December in conjunction with the ITER Ministerial meeting. Ours was the only talk based on fusion and was highlighted as the first talk at the Workshop. The concept, summarized in a White Paper that was submitted by the U.S. to the ITER organization, was so well received that the it is being discussed as the model for organizing the worldwide scientific collaborations required to make ITER a success.

The DIII-D Research Opportunities Forum was broadcast via AG technology for the first time. Participants were worldwide including England and Germany. Although successful, the broadcast highlighted important audio considerations for this conference room that will also be applicable to the weekly DIII-D science meeting that will be broadcast using the same technology.

The Project web site continued to be updated as required.

Security/Remote Computing

The prototype end-to-end agreement-based service providing QoS, for between-pluse execution of fusion applications, was made more robust this quarter. This work included code streamlining for improved performance through implementing service precreation. The finished system combined network transport prediction with application runtime prediction to give the end user an end-to-end estimate of the execution time of the EFIT application. Notification of the progress through the different execution stages was integrated with DAM. This between-pulse computation was demonstrated at SC2003.

A prototype graphical tool to display all Akenti policy and use conditions for a resource has been completed and is ready for user evaluation. The deployment of the Akenti-Job manager code at PPPL for TRANSP was delayed this quarter due to other higher priority items related to the collaborative control room demonstration.

To support the SC2003 demonstration, a new MDSplus server was created to mimic an actual DIII-D shot cycle including events and data acquisition. This capability will also be used in the future, as more demonstrations of this capability will be given at a variety of meetings.

The NFC Project began to investigate the new Globus XIO features in the MDSplus client/server data access services. This has the potential for several benefits including higher performance communication for transferring large amounts of data over high bandwidth, high latency network connections. The need for this communication improvement has been highlighted by our previous work with the large NIMROD simulation datasets.

Visualization

For the SC2003 demonstration, VenueVNC was updated to be compatible with AG 2.1.2. New settings were determined to allow for higher compression for more fluid interaction under both Windows and Linux. These updates allowed for easier sharing of the fusion visualization tool ReviewPlus between Phoenix and San Diego.

The small AG node (PIG) was permanently installed in the DIII-D control room in preparation for the SC2003 demonstration and the start of DIII-D tokamak operations. A web-based camera (Sony SNC-RZ30N) that is externally controllable was installed in the DIII-D to give the scientists the ability to look around the control room. This camera video is also available via the AG node and was made available for the SC2003 demonstration.

Based on user feedback, a web interface was created to give off-site researchers information on the state of the pulse cycle, the status of data acquisition, and the state of between pulse data analysis. The ElVis applet was implemented to give the off-site researcher the ability to do graphical data monitoring, to complement the traditional FGM system, of TRANSP computations on FusionGrid. This implementation is general such that any FusionGrid service can use this technology. This capability was also part of the SC2003 demonstration.

Based on the feedback from last quarter's testing, a 3-tile display wall was designed for DIII-D and the equipment was purchased (Toshiba P500DL display cubes) with non-NFC funds. Installation will take place next quarter and in time for DIII-D operations. In a similar fashion, the 2-tile NSTX display wall as designed, reviewed, and installed and initial testing has begun.

SCIRun macro modules were released and used by the NIMROD team for easier use of the visual programming interface. Results of this work were used at SC2003 and also for a DOE presentation by the NIMROD team.

Appendix A: Non-Edited Reports from Individual Institutions

A.1 M. Papka for the ANL MCS, Futures Laboratory

Major efforts for the past quarter were focused around Supercomputing 2003 (SC), ITER demonstration in Washington D.C., and in preparation for proposal renewal. In preparation for SC, ANL/FL participated in weekly planning meetings, handled all logistics for shipment and setup/breakdown of demonstration space at SC. In preparation for the ITER demo ANL/FL downloaded and converted HD video footage recorded at SC in hopes of using it during the preparation, participated in conference calls, tested with Access DC AG node, and attended Saturday meeting. Preparing for the FusionGrid renewal ANL/FL participated in planning sessions in December.

Particular efforts included the following:

• Construction of a wxPython-based set of application wrappers for the command line tools used in requesting reservations and subsequently claiming them. This provided a means for retrieving and displaying the information returned by the system, and a means for providing all the necessary arguments during both the request and claiming phases.
• VenueVNC was updated to be compatible with AG 2.1.2. New settings were determined to allow higher compression for more fluid interaction under both Windows and Linux. Worked with with General Atomics (GA) to get ReviewPlus working better under VNC, this included:
• Using a newly updated ReviewPlus that had auto update enabled
• Modified VNC settings so that fonts appear properly in ReviewPlus and color maps were closer to what was expected by the user
• Worked with GA to get a EFIT data viewer program working under VNC
• Tested with GA on multiple occasions
• Worked with Princeton Computer Science (PCS) to get their modified window manager working on SC demonstration machine and under VNC; this included installing a modified version of wm2, providing debugging feedback to PCS, testing auto update enabled ReviewPlus EFIT viewing under PCS modified window manager
• Worked with PPPL to get their Access Grid node fully operational, including helping to get AG 2.x installed, debugging some of their audio problems, and helping to get VenueVNC installed and working.

A.2 K. Keahey for the ANL MCS, Distributed Systems Laboratory

In the last quarter we made contributions in the following areas:

1. Work on agreement-based application servers
1. Our main focus this quarter was the continued work on a prototype of an end-to-end agreement-based service providing QoS for between-pulse execution of fusion applications. In this quarter we focused on making the prototype more robust with a view to demonstrating it at SC03. Accordingly, we streamlined the code and improved its performance through implementing service precreation. We installed it at the Fusion labs and integrated it with fusion applications. The finished system combined network transport prediction with application runtime prediction to give the end-user an end-to-end estimate of the execution time of the EFIT application. Notification of the progress through the execution stages of the application was integrated with the Data Analysis Monitor (DAM) displayed over the Access Grid. The finished between-pulse application was demonstrated at SC03.
2. We contributed the requirements and experiences of this project to the work on WS-Agreement specification taking place at the Global Grid Forum. This effort will lead to the development of Grid standards, eventually enabling Fusion scientists to obtain agreements for QoS-based execution of their codes.
3. We helped organize and participated a meeting demonstrating Grid and collaborative capabilities that could be of use to the ITER project. The demonstration took place at the December ITER meeting and was highly successful.
2. Work on authorization and enforcement
1. We continued to assist our collegues at LBNL with integrating Akenti authorization into the GT2 GRAM authorization package. We continued work on including this package in the Globus 3.2 due to be released shortly.
2. We presented the outcome of our work on dynamic accounts: a paper describing it has been presented at the Designing and Building Grid Services workshop held in conjunction with GGF9 (October) and a poster describing this work has been presented at SC03 (November)
3. Other accomplishments
1. Firewalls. We organized a panel of site administrators and Grid Security experts to brainstorm specific solutions that would be required for the solution of the firewall problem in Grids to be held at the Globus World conference in January 2004.
2. We provided technical support, assisted with debugging and troubleshooting of Globus installation and associated technologies. This involved providing explanations and technical discussions of Globus functionality as well as providing examples, help with troubleshooting and answering questions.
3. We also interacted with our collaborators through frequent teleconferences, AG meetings, discussions, etc.

A.3 D. Schissel for the General Atomics Fusion Group

General

• Coordinated and participated in SC03 demonstration on the collaborative control room where off-site scientists (SC03 show floor) remote participated in DIII-D experimental operations (see Fig. 1) (Schissel).
• Participated in the APS/DPP meeting and organized the NFC Oversight Committee meeting held at the same venue (Schissel).
• Presented the Collaborative Control room concept at the ITER Grid workshop help in conjunction with the December ITER Ministerial meeting (Schissel).
• Equipment was purchased using non-NFC funds for three additional AG nodes to support Fusion research at the DIII-D National Fusion Facility (Schissel).
• Three Toshiba P500DL display cubes were purchased using non-NFC funds for the DIII-D control room to create a tiled display to support experimental operations (Schissel).
• The DIII-D Research Opportunities forum was broadcast using AG technology and off-site participants include ORNL, MIT, JET fusion facility in England, and the ASDEX fusion facility in Germany (Schissel).
• The project web site (http://www.fusiongrid.org) was maintained (Schissel).

Security/Remote

• Support was provided for the ANL work on preemptive scheduling and QoS issues for running TRANSP between pulses for SC03 demo (Peng).
• Preparation for the SC03 demonstration including creating a new MDSplus server to mimic an actual DIII-D shot cycle including events and data acquisition (Peng).
• Worked with Kruger and Sanderson on soft X-ray data preparation allowing the comparison of NIMROD simulation data with actual DIII-D data (Peng).
• Support was provided for the ANL work on running EFIT on the PPPL cluster for the SC03 demonstration (Peng).
• Initial testing on Globus Version 3 (GT3) began this month to investigate what will be required to move to this new version (Burruss).

Visualization

• The small AG node (PIG) was permanently installed in the DIII-D control room for usage in the SC03 demonstration and in preparation for DIII-D operations early in calendar year 2004. (Abla).
• A Web interface was created to give off-site researchers information on the state of the pulse cycle, the status of data acquisition, and the state of between pulse data analysis (Peng, Abla)
• A Web based camera (Sony SNC-RZ30N) that is externally controllable was installed in the DIII-D control room to give off-site scientists the ability to look around as if they were physically present (Abla).

A.4 M. Thompson for the Lawrence Berkeley National Laboratory

PPPL took over the deployment of the Akenti-Job manager code this quarter, but due to other priorities the deployment was not completed. The system administrator wrote the policy and use-condition files for Akenti authorization and tested them against a standalone Akenti access checker. The policy design allows new users to be given access by simply creating one attribute certificate which adds them to an appropriate group. Likewise, a user can be denied access, by removing a single attribute certificate.

LBNL finished a graphical tool to display all the policy and use-condition certificates for a resource. This tool enables stakeholders to easily see all the policy in an interactive way.

The Akenti-Globus job manager code was used as part of the LBNL SC03 demo of a completely monitored remote job submission. We instrumented a pyGlobus job submission client, the Globus gatekeeper and job-manger and the Akenti authorization code with NetLogger write statements. All the logging output was collected by a NetLogger Visualization server which graphically displayed a life line for the job. If at any point the job failed, the output can be examined in detail to determine cause of the failure.

We had more discussions about the difficulties of certificate management, culminating in a decision to provide script interfaces as an alternative to the web based methods of requesting and retrieving certificates. I have begun negotiations with the DOEGrids CA administrators to test interfaces that the scripts can use.

A.5 M. Greenwald for the MIT Plasma Fusion Science Center

The Fusion Grid team members at MIT explored the potential use of the new Globus XIO features in the MDSplus client/server data access services. The Globus XIO simplifies the interface for doing both unsecure and secure communications and will also eventually provide a simple way to add high performance, striped socket communications for transferring large amounts of data over high bandwidth, high latency (long distance) network connections. MIT worked with the Globus XIO developers for specifying and testing a suite of sample applications which demonstrate the types of communication protocols need by the MDSplus system.

MIT has also started investigating simple OpenSSL secure communication methods as possible lightweight solutions for providing secure authentication for MDSplus on systems for which the full Globus functionality is either not available or not needed.

A.6 D. McCune for the Princeton Plasma Physics Laboratory

Collaboratory Computational Services:

• Feedback to Globus/DOEGrids people:
• Report to DoeGrids on our Grid Administration experience. Subjects concerned use of certificates, ease of installation and patching by users and administrators; firewall issues, and security of X509 user keys.
• Important information based on experience from production use of the system.
• Support of between-shots TRANSP and parallel EFIT:
• New more powerful computer added to PPPL cluster for time-critical jobs.
• Globus toolkit 3 with modifications by Argonne for advance scheduling and job pre-emption supported for acceptance of time critical jobs from off-site at PPPL (tests were carried out).
• Demonstrated at Super Computing 2003.
• Number of TRANSP users and sites:
• IPP/Garching (Germany) now has Globus installed to enable use of the NFC TRANSP service by their users on the ASDEX-U tokamak.
• Regular usage by JET and MAST as well as DIII-D, Cmod and NSTX now seen.
• Total number of TRANSP users with NFC/DOEGrids certificates now at 50.

Collaboratory Visualization Activity:

• NSTX Control Room collaborative display installed and tested.
• Design reviewed with NSTX experimental physicists
• Projectors, screen, computers and networking installed and tested.
• Implemented ElVis applet for data monitoring.
• Client software runs in user web browser.
• Deployed in TRANSP production system to give visual access to input data and select simulation results during job execution.
• Integrated with GA FGM -- access by click on link under TRANSP run id.
• Integrated with shared X tool (PCS).
• Presented and demonstrated at SC03.
• Installed Access Grid (AG) software v2.1.2 on PPPL Display Wall:
• Worked with ESnet engineers to debug multicast packet loss. ESnet enabled fast switching on their router and this eliminated the packet loss.
• Audio buffer overflow / crash problem identified in collaboration with ANL engineers: hardware compatibility issues -- ordered replacement sound board.
• AG used for PPPL participation in SC03 demo, and for attendance at SC03 AG-hosted seminars.
• AG used to enable PPPL physicist participation in DIII-D experiment planning meeting; physicist feedback on current AG strengths and weaknesses provided.

A.7 K. Li for the Princeton University Computer Science Department

No report.

A.8 A. Sanderson for the U. of Utah Center for Scientific Computing & Imaging

In the first quarter SCIRun was released with macro modules which makes it easier to use the visual programming interface for creating visualizations. This was immediately put into use for supporting the NIMROD group in visualization data for Super Computing and in their DOE report for their big splash allocation.

During this same time, time was spent harding SCIRun for support of large time dependent simulations such as those with NIMROD. This hardening allowed data set of over 1G is size to be animated in SCIRun.

The final area of major work was support for axis in both 2 and 3 dimensions which makes it possible for qualitative plots to be made. With this addition both experimental and theoretical researchers can use SCIRun as a day to day tool.

This complete package is now available as a package for Fusion Researchers.