The following report is available in PDF format for high quality printing.

First Quarter FY 2002 Report – The National Fusion Collaboratory

Edited by D.P. Schissel1

1General Atomics (schissel@fusion.gat.com)

Overview

This report represents the start of the National Fusion Collaboratory Project. Receipt of funds by the six institutions spanned a several month interval. The earliest arrival of funds was in August and the latest was in early October 2001. Given this spread of funding it has been decided to have this report summarize work up to December 31, 2001 and call this the First Quarter FY 2002 Report.

General accomplishments include:

·             The registration and creation of a project web site (http://www.fusiongrid.org)

·             The creation of project mailing lists for participants and the oversight committee

·             The creation of a detailed working plan including review by the oversight committee

·             A secured remote computation with visualization at SC01 that validates our Grid design

·             Display wall prototypes of X–windows, shared visualization, and an AG node

·             The submission of a paper on Computational Grids and the Fusion Collaboratory

General

A project web site (http://www.fusiongrid.org) was registered and created to satisfy both the needs of those individuals working to create the Collaboratory and the scientists who will hopefully benefit from the work. The workers portion of the web site has restricted access and contains a detailed work plan, email archive lists, and detailed plans on future meetings and Collaboratory demonstrations.

An initial meeting of the oversight committee was held during the APS/DPP fusion science meeting in Long Beach. This annual meeting of the U.S. magnetic fusion community provided an excellent opportunity to discuss the goals and initial working plan with the customer. Feedback from the oversight committee and the user community was valuable and folded into the working plan that includes demonstrations of new capability at two distinct fusion science meetings in April 2002. Detailed discussions with DOE (Bair, Scott) included a meeting at General Atomics and at SC01 in Denver, participation in the AG node PI’s meeting, and the general SciDAC meeting at SC01. Meetings of the Collaboratory partners included the Security/Remote working group in Chicago and Denver, and the Visualization working group in San Francisco and San Diego. Both groups also communicated via email and conference calls. To help familiarize individuals with the science of magnetic fusion a number of Collaboratory team partners visited the DIII–D National Fusion Facility this quarter (Bair, Finkelstein, Hansen, Johnson, Papka, Scott, Stevens). To facilitate coordination among other SciDAC projects a discussion was held with Deb Agarwal in San Diego on potential synergy between her projects and the Fusion Collaboratory.

Presentations on the Collaboratory took place at several satellite meetings during the APS/DPP meeting, at WACE 2001, at the Globus Retreat, and at the DOE/OFES PSACI PAC meeting. A demonstration by the Fusion Collaboratory of secure distributed computing was presented at SC01. A paper was submitted to the Journal of Future Generation Computer Systems on Computational Grids and the Fusion Collaboratory.

Security/Remote Computing

The effort this quarter of the Security/Remote working group culminated in a demonstration of secured remote computing at SC01 in Denver in both the ANL and LBNL booths. The demonstration illustrated a typical interaction of fusion applications in a Grid environment. The demonstration consisted of four components deployed in remote locations and interacting through a Globus-enabled implementation of the MDSplus framework: (1) a show–floor controller program orchestrating the interactions of the other components, and interacting with the user, (2) an LBNL MDSplus data server, (3) the EFIT a parallel analysis code on an ANL CPU cluster, and (4) scientific visualization of the data on the show–floor. While the prototype system lacked several desired features, it did demonstrate that the technology exists to add a significant level of security to the remote access capability of MDSplus without sacrificing functionality or performance and therefore provided a proof of concept that our design for Grid-enabling fusion applications is correct.

The main capabilities that we demonstrated were multi-site security and uniform access to remote computational capabilities. Both the MDSplus server code (mdsip) and the MDSplus client API (mdsipshr) were fitted with globus_io calls. Adding calls to Globus IO in MDSplus lead to the development of general-purpose functionality that will be used to Grid-enable other fusion applications and is an important step towards building the Fusion Grid. When a client using this modified software attempts to connect to an MDSplus server on a remote system, the client sends secure credential information to the server and the server sends additional credentials to the client and both the client and the server determine if the communication between them is authorized. When the authorized link is established, subsequent communication between the client and server is encrypted to maintain a high level of security. As part of this effort, the LBNL Grid CA was enabled to issue x509 certificates to Fusion partners for use as Globus compliant certificates. Additionally, the CA signing policy and Grid Map files were modified on several nodes in the initial Fusion Grid to accept Fusion partner certificates. With this initial proof of concept completed, work began on creating a Grid enabled 42–node Linux cluster for U.S. TRANSP computations. This will represent a significant step beyond the EFIT capability demonstration of SC01 since these services will have a limited release to the user community for testing and feedback.

In addition to the MDSplus/Globus activities, work has begun to add the same authentication, authorization and encryption security to remote access to Microsoft SQL relational databases used in fusion research. Extensions to MDSplus have been added to enable access to relational databases using the MDSplus client/server communications mechanism. Although remote access to these databases can be channeled through secure MDSplus client/server connections, the port of Globus to the Windows operating system will still be required for a complete implementation. The security/remote computing working group will continue to work with the Globus group to insure a timely completion of this port to the Windows operating system.

A paper was submitted to the Journal of Future Generation Computer Systems. The paper summarizes the requirements of fusion applications in the Grid context, provides an overview of relevant technologies (Globus and Akenti), describes our preliminary work as implemented in the SC01 fusion demo, and outlines the future plans of the Collaboratory.

Visualization

The visualization working group supported the demonstration at SC01 by creating images and movies with the real time ray tracer showing experimental time–varying isosurfaces in magnetic flux space and colored by plasma temperature. This required preprocessing the data file to convert the 2D experimental magnetic flux data into a swept surface on a 3D regular grid, mapping the experimental temperature data to that grid, and modifying the real time ray tracer to color the magnetic flux isosurface based on the corresponding temperature for a particular time step. The images were used in both the SC01 poster and handout and the movies were shown as part of the demonstration illustrating the results of the remote computation.

The quarter was also spent familiarizing the computer science partners with the visualization needs of the fusion scientist. This work involved the installation of MDSplus at several laboratories as well as the installation of several IDL based visualization tools that are used through the fusion community. This education will be an evolving process and will involve additional work to complete these installations so that the new visualization codes read data directly from MDSplus and not through an intermediate file.

Another thrust of work this quarter was experimentation with visualization on multiple tiled display walls at ANL and the linkage of walls at PCS and PPPL. If successful, this work could pave the way for the connection of these walls among different tokamak control rooms within the fusion community. Initial tests of OpenGL with the Stanford Chromium library were positive and warrant further investigation. However the present results only allow application control from one wall. Other tests of simultaneous visualization on two display walls included the use of WireGL to stream OpenGL 3D visualizations, the use of the Java based Scivis tool to render 2D graphs, development of an OpenGL based graphics library for use with the Chromium library, and the use of VNCWall to render and manipulate an X–windows desktop. The results of these tests will be discussed among the visualization working group in the second quarter and a consensus will be reached on a future course for the Collaboratory.

Work this quarter also involved examination of extensions to the Access Grid. Extension classes to VTK allowed for streaming of H261 to a video conferenceing (vic) tool and its subsequent integration with the AG. The vic tool was also modified to allow for the interaction between all participants and the host application. Utilizing the Chromium library, software was created that allows any Chromium aware OpenGL application to be demonstrated on the Access Grid. This represents a scalable test–bed application for delivery of high–resolution images in the AG environment. This work has allowed users of the AG to share visualization applications and therefore explore the collaborative visualization space. The Xplit environment has been prototyped using the NCSA modified Virtual Network Computer (VNC) software to demonstrate what an AG node would be like on a tiled display wall. This prototype allows fusion specific X–windows applications, such as those written in IDL, to be run in this environment. Although this work on AG node technology represents an important step for the Collaboratory, it is recognized that possibly only a few U.S. fusion sites will have an AG node during the initial three–year phase of the Collaboratory. Therefore, examination of remote visualization and participation in AG conferences from fusion sites with “simple” hardware capability will continue to be investigated by the visualization working group.

Appendix A: Non–Edited Reports from Individual Institutions

M. Papka for the ANL MCS, Futures Laboratory

·             MDSplus client installed – this was done in order to have access to sample DIII-D data and for use with ReviewPlus. Currently not fully operational due to some configuration issues. ANL is getting GA employees accounts on ANL machines in order to fix the problem.

·             ReviewPlus & IDL installed – ReviewPlus is a tool used currently by the experimentalists to analysis data, purpose for installing is to evaluate what tools are currently being used by the experimentalists, first to propose new improved tools, followed up by prototypes. ReviewPlus is currently working but needs access via MDSplus to look at actual datasets.

·             Work with Stanford’s Chromium Library –

o           An initial test determined if one could use an OpenGL application with the Chromium library to drive multiple tiled displays. This proved successful allowing users at remote sites to view the same high resolution images on tiled displays. Current drawback only operates in a tour mode, which means only the site where the original OpenGL application is running has control.

o           Development of RenderToH261 SPU that is derived from the standard Render SPU for the chromium library, this allows for any OpenGL application that works with chromium become an application that can be demonstrated on the Access Grid. By coupling this SPU with the standard tile sort SPU from Chromium ANL has developed a scalable testbed application for the delivery of high resolution images in the Access Grid environment.

Single H261 stream (352x288) consumes ~1.1Mb/s @ 15fps

Six H261 streams (1056x576) consumes ~2.7Mb/s @ 15fps

Figure 1 Desktop image of visualization application distributed across 6 video streams.

·             The development of extension classes to the Visualization Toolkit (vtk) for the streaming of H261 to Video Conferencing (vic) tool and integration with the AG. The vic tool was also modified to allow for interaction between all participants and host application. This work is has allowed users of the AG share visualization applications and ANL to explore the collaborative visualization space. This work is a precursor to the work discussed above using Chromium which can duplicate the extension classes and allow for multiple tiles to be streamed.

·             Development of an OpenGL based 2D graphing library, for use in with other applications or for building stand-alone tools. Care has been taken to allow it to work with the Chromium library for ease of use in tiled display environments.

Figure 2 Output of OpenGL graphing library, 4 graphs layered together.

·             Prototyping of the Xplit environment using VNC, ANL has used the Virtual Network Computer (VNC) software from AT&T and enhanced by NCSA to prototype what an AccessGrid Node would be like on a tiled display environment. This included running standard AG applications such as vic and web browsers along with Fusion specific applications such as IDL and ReviewPlus.

 

K. Keahey for the ANL MCS, Distributed Systems Laboratory

 

 

 

 

 

 

D. Schissel for the General Atomics Fusion Group

General

·             A project web site (http://www.fusiongrid.org) was registered and created to satisfy both the needs of those individuals working to create the Collaboratory and the scientists who will hopefully benefit from the work. This work included the reconfiguration and relocation of an existing Fusion Group Linux workstation for exclusive use as a web server. The workers portion of the web site has restricted access and contains a detailed work plan, email archive lists, and detailed plans on future meetings and Collaboratory demonstrations.

·             An initial meeting of the oversight committee was held during the APS/DPP fusion science meeting in Long Beach. This annual meeting of the U.S. magnetic fusion community provided an excellent opportunity to discuss the goals and initial working plan with the customer. Feedback from the oversight committee and the user community was valuable and folded into the working plan.

·             Detailed discussions with DOE included a meeting with DOE (Bair, Scott) at General Atomics and at SC01 in Denver, participation in the AG node PI’s meeting, and the general SciDAC meeting at SC01.

·             Participated in meetings of Collaboratory team members including the Security/Remote working group in Chicago and Denver, and the Visualization working group in San Francisco and San Diego. Communicated with both groups via email and conference calls.

·             Hosted visits to the DIII–D National Fusion Facility to help familiarize individuals (Bair, Finkelstein, Hansen, Johnson, Papka, Scott, Stevens) with the science of magnetic fusion. To facilitate coordination among other SciDAC projects a discussion was held with Deb Agarwal in San Diego on potential synergy between her projects and the Fusion Collaboratory.

·             Presentations on the Collaboratory were given at several satellite meetings during the APS/DPP meeting, at WACE 2001, and at the DOE/OFES PSACI PAC meeting.

Security/Remote

·             Participated in the design and creation of the Collaboratory demonstration at SC01 by creating a parallel version of the EFIT code that writes data to an MDSplus data repository. Installed and tested this software on the ANL computational cluster and wrote data to the LBNL MDSplus server.

·             Designed the poster and handout material used at SC01.

·             Participated in the creation of the working plan including the demonstration of new capability in April 2002.

·             Contributed to a paper on Computational Grids and the Fusion Collaboratory that was submitted to the Journal of Future Generation Computer Systems.

Visualization

·             Supplied data to be visualized for the SC01 demonstration and assisted in the design of these visualizations.

·             Facilitated the specifications for initial visualization work for NIMROD data.

·             Participated in the creation of the working plan for initial visualization tasks including demonstration of new capability in April 2002.

M. Thompson for the Lawrence Berkeley National Laboratory

The major effort in this quarter was preparing for a demonstration of a basic fusion analysis process taking place on several distributed Globus nodes with the legacy fusion codes using secure Globus communication channels. Most of the bullets below represent steps in that effort. See http://www.fusiongrid.org/about/news/sc2001.html for a detailed description of the demo.

The main contribution of the demo to the work of the collaboratory was to give us practice is setting up Globus nodes, getting and using X.509 certificates for secure communications, and learning how to add secure Globus communication to legacy codes. The contribution of LBNL was to provide early access to the established DOE science grid nodes, issue X.509 certificates to fusion members, and to share our experience in settingup Globus nodes and writing Globus applications with the fusion physicists that were globizing the two legacy fusion codes, MDSplus and EFIT.

·             Enabled the LBNL-Grid-CA to issue x509 certificates to Fusion Members for use as Globus compliant certificates.

·             Set the CA-signing-policy and Grid-Map files on the DOE Grid nodes at LBNL to accept the fusion member’s certificates and made several LBNL nodes available for debugging and running the globized version of MDSplus.

·             Installed Globus 2.0 and the fusion credentials on a Sun workstation that was taken to SC01 as part of Fusion collaboratory demo.

·             Presented the demonstration of an analysis of real-time fusion shot analysis communicated over secure Globus i/o at SC01 in Denver, Nov 10-16.

·             Contributed input to the poster and handouts used at SC01.

·             Presented the overview of the Fusion Collaboratory at the Globus Retreat following HPDC10 in San Francisco Aug 10. This was one more opportunity to inform the Grid community of the existence and goals of the National Fusion Collaboratory.

T. Fredian for the Massachusetts Institute of Technology Plasma Fusion Science Center

During this period significant progress was made in prototyping the addition of secure authentication and authorization to the remote access capabilities of the MDSplus (see http://www.mdsplus.org) data handling system used widely in fusion energy research. MDSplus provides scientists with the ability to access fusion research data located essentially anywhere accessible via the Internet. A scientist can run an analysis program that pulls in data from numerous fusion experiments anywhere in the world and write the analysis results into a remote data store. As the use of this capability has grown it has become increasing apparent that a sophisticated method of providing secure authentication and authorization was needed to ensure the integrity of the research data was maintained and access to it was limited to authorized users. In pursuant to this goal, the client/server communications used in the MDSplus remote access services were converted to use Globus (see http://www.globus.org) I/O services. Globus provides secure authentication and authorization based on pass phrase protected X509 certificates. Communications between the client and server can be further secured by data encryption.

Accomplishment details

Both the MDSplus server code (mdsip) and the MDSplus client API (mdsipshr) were fitted with globus_io calls. When a client using this modified software attempts to connect to an MDSplus server on a remote system, the client sends secure credential information to the server and the server sends additional credentials to the client and both the client and the server determine if the communication between them is authorized. If it is then the communication link between the two processes is opened and subsequent communication between the client and server is encrypted to maintain a high level of security.

This prototype system was demonstrated at the November Super Computing 2001 conference in Denver, Colorado. A controller program running at the conference in Denver sent a message to a cluster of computers running at the Argonne National Laboratory to start an analysis program. This analysis program analyzed some fusion data and then used a secure MDSplus/Globus connection to an MDSplus server at the Lawrence Berkeley National Laboratory to write the results into an MDSplus data store. The completion of this analysis was report via another secure MDSplus/Globus connection to the controller program running at ANL, which in turn activated a graphical display program. This display program accessed the analysis results again using a secure MDSplus/Globus connection from ANL to the data stored at LBL.

The goal to demonstrate this secure MDSplus/Globus communication capability at the SC2001 conference was extremely ambitious and required a coordinated effort of team members at MIT, ANL, LBL and GA. Numerous conference calls were required to layout the demonstration scenario and to coordinate the development effort. The demonstration ran successfully and was presented at two separate booths at the conference, an ANL boot and an LBL booth. This was quite impressive given the scope of the demonstration and the extremely short time frame to complete the prototype and assemble the demonstration.

While the prototype system lacked several desired features, it did demonstrate that the technology exists to add a significant level of security to the remote access capability of MDSplus without sacrificing functionality or performance.

In addition to the MDSplus/Globus activities, work has begun to add the same authentication, authorization and encryption security to remote access to relational databases used in fusion research. Extensions to MDSplus have been added to enable access to relational databases using the MDSplus client/server communications mechanism. Eventually all remote access to these databases can be channeled through secure MDSplus client/server connections thus providing a high level of security.

D. McCune for the Princeton Plasma Physics Laboratory

The focus is on creating a TRANSP (http://w3.pppl.gov/transp) computation service with Collaboratory (i.e. network) capability, using MDSplus for the data networking. This will also be the TRANSP service used at PPPL, which will replace the legacy VMS/UNIX master/slave production system. There are two thrusts to this work:

Application software (Tina Ludesher)

System software (Lew Randerson)

G. Wallace the for Princeton University Computer Science Department

PCS-PPPL collaboration

 

Regarding SOW sections 3.2.2 and 3.2.3; work on collaborative remote visualization using high definition displays.

o           Verified network access through 100Mb microwave link

o           Setup temporary hole in PCS firewall for display wall communication

o           Tested bandwidth and latency (unsatisfactory – needs tuning)

o           WireGL to stream OpenGL 3D visualizations to both walls. Application run from client at PPPL.

o           Scivis to render 2D graphs simultaneously. Application run from PPPL.

o           VNCWall to render and manipulate an Xwindows desktop simultaneously on both walls.

 

PPPL initiative

o           We have made it work with MDS+ framework

o           Work with TRANSP(http://w3.pppl.gov/transp/) requirements.

§             Modification of API’s for better support of Multiplots

§             Additions to the API’s to support better label support.

o           Changes from Java 1.1.x to Java 1.3.x

o           We are starting to make scivis into a 3-tier architecture. This is important so that users will only install open-source clients on his/her workstation/wall.

o           We are starting to work with portals, so that scivis services can be put onto this portal. We want RPLOT (http://w3.pppl.gov/transp/MDSPLUS/) to be one such service on our portal.

o           We can’t run Opengl within VNC Wall.

C. Johnson for the University of Utah Center for Scientific Computing and Imaging

Work at Utah since the inception of the SciDAC Fusion Collaboratory has primarily focused on visualizing experimental and, more recently, simulated fusion data from scientists at General Atomics.

Specific Contributions

·             We wrote preprocessing programs to convert the 2d experimental flux data into a swept surface on a 3d regular grid, as well as, programs to preprocess the experimental fusion temperature data.

·             We wrote code for the real time ray tracer (rtrt) to read this 3d flux data to take advantage of the isosurface extraction capabilities already included in the rtrt.

·             We wrote code for the rtrt that would color the flux isosurface based on the corresponding temperature for a particular time-step.

·             We created images (Fig. 3) and movies with the rtrt showing experimental time-varying (see http://www.sci.utah.edu/sci_images/images13.html) isosurfaces based on flux and colored by temperature. The images were used for a Supercomputing 2001 poster and the video was used in a SciDAC Fusion Collaboratory display at SC 2001.

·             In October, the Utah visualization team visited General Atomics and met with their computational fusion scientists to learn about their visualization needs.

·             On December 10, 2001, Allen Sanderson joined the Utah Fusion Collaboratory visualization team. Allen has a Ph.D. in Computer Science from the University of Utah. Allen has expertise in computer graphics, modeling, image processing, and visualization. Prior to joining the SCI Institute, Allen worked at Evans and Sutherland Computing.

·             While still in the process of getting up to speed, Allen has started to establish the code necessary to connect to MDSplus via the C API.

·             We are hosting Scott Klasky (from PPPL) on Jan 17-18 and giving him a tutorial on our current visualization software.

·             We are hosting David Schissel (from GA) on Jan 21 to get feedback on our visualization work to date.

Figure 3 Visualization of temperature mapped to magnetic flux surfaces

 

This report is available in PDF format for high quality printing.

team contact info | about the fusion grid | fusiongrid research

Last modified 01/11/02. Comments? webmaster