Upload
marybeth-francis
View
218
Download
0
Embed Size (px)
DESCRIPTION
The Grid The Grid is an abstraction that allows transparent use of distributed computing resources. The problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations.
Citation preview
VisPortal Projectdeveloper’s experience
C.E.Siegerist, J. Shalf, E.W. BethelNERSC/LBNL Visualization Group
T.J. Jankun-Kelley, O. Kreylos, K.L. MaCIPIC/UC Davis
Overview• Grid Introduction • Portal Architecture/Software
components. • Visportal Services/Example Pages• User’s perspective• Conclusions/Future/Caveats• References
The Grid• The Grid is an abstraction that allows
transparent use of distributed computing resources.
• The problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations.
VisPortal Project• The LBNL/NERSC VisPortal effort explores
ways to deliver Grid-based advanced visualization and data analysis capabilities through a Web Portal interface.
• Using standard Globus Grid middleware and off the shelf web automation, the VisPortal hides the underlying complexity of resource selection and distributed application management.
VisPortal ServicesMotivation: simplify remote visualization, offer point an click solutions for complex distributed applications
• File Management• Vis Applications: Visapult, AMR Renderer,
Web VisSheet.• One button Mpeg• …
Globus• The Globus Project provides software
tools (Globus toolkit) that make it easier to build computational grids and grid-based applications.
Globusrun Example of a file conversion RSL script
% globusrun –r escher.nersc.gov –f convert.rsl -w
& (executable=$GLOBUSRUN_GASS_URL # “/home/cristina/convertscript”)
(enviroment=(GLOBUSRUN_GASS_URL $(GLOBUS_GASS_URL))
#!/sbin/sh GLOBUS_LOCATION=/usr/local/globusexport GLOBUS_LOCATIONPATH=$GLOBUS_LOCATION/bin:$GLOBUS_LOCATION/sbinexport PATHLD_LIBRARY_PATH=$GLOBUS_LOCATION/libexport LD_LIBRARY_PATHIMAGE_MAGICK=/usr/freeware/bin export IMAGE_MAGICKglobus-gass-server -t -u -r -w -c > $mytmpgassfile &sleep 3set remote_gass_server_url=`cat $mytmpgassfile`rm $mytmpgassfile $GLOBUS_LOCATION/bin/globus-url-copy $GLOBUSRUN_GASS_URL/home/cristina/file.jpg file:/tmp/file.jpg$IMAGE_MAGICK/convert /tmp/file.jpg /tmp/file.bmp$GLOBUS_LOCATION/bin/globus-url-copy file:/tmp/file.bmp $GLOBUSRUN_GASS_URL/home/cristina/file.bmpglobus-gass-server-shutdown $remote_gass_server_url rm /tmp/file.bmprm /tmp/file.jpg
GPDK• The Grid Portal Development Kit (GPDK)
is a set of modular, reusable components for accessing Grid services in the form of Java Beans. It derives most of its functionality from the Globus Java Commodity Grid (Java CoG) toolkit.
• Other toolkits: GridPort, GridSphere
Portal ArchitectureHPC
Resources
WebPortal
Parallel VisualAnalysis Server
User client 1
Archival Storage
& Data Cache
User client n
User site(s)HPC Facility
Audio& video
DataGRAMGSIMDS
GRAMGSI
GSIFTP
httpsSOAPGridFTP
CoG
Portal Architecture (thick client)
HPCResources
WebPortal
Parallel VisualAnalysis Server
User client 1
Archival Storage
& Data Cache
User client n
User site(s)HPC Facility
Audio& video
DataGRAMGSIMDS
GRAMGSI
GSIFTP
httpsSOAPGridFTP
CoG
Thick Client
Resources | Users
Internal ArchitectureWeb Server
Tomcat
GPDKPortal Engine
(Servlets)Application Logic Presentation
(JSP)
Grid Service BeansSecurity Job
SubmissionInformation
ServicesData Transfer
User Profiles
Grid Middleware Libraries(e.g. CoG)
globus
Software Components
• Apache 1.3.27• Tomcat 3.3.1• gpdk cvs version• Java CoG v1.0a• mod_jk 1.3
• mod_ssl 2.8.12
• openssl 0.9.6h • globus 2.x• java 1.4.x• MyProxy 1.0• log4j_1.2.7
GPDKDemo Portal
Login Page
• Myproxy: username and password of delegated credential
• Local: load from credential file (grid-proxy-init)
• Update user’sresourcesjob historyemail address
Profile
File Mgmt.
• Foreground transfers
• Background transfers
• Globus File Yanker(Shreyas Cholia)
Launch ApplicationsVisapult
Launch ApplicationsAMR Renderer
Visportal WebSheet
MPEG Generator
• Example of a user driven service: the user connects to our Onyx to run an IDL batch job to produce a movie with data that he has in an IBM SP • The portal provides an interface to select the data file and the mpeg parameters. It transfers the file, launches the IDL job and sends an email to the user with a URL to retrieve the movie.
The portal is running (most of the time)
https://frost.lbl.gov/visportal/servlet/visportal
User’s perspective• Accounts in all the resources.
• Request a user certificate: http://www.doegrids.org/pages/cert-request.html
• Be in the grid-map files of the resources (automated through NIM).
• Create a proxy certificate and login.
Challenges
• Difficult GUI Implementation• Error Recovery• Support• Middleware is a moving target• Resource management
Future• Deploying a subset of services to
NERSC users.• Implementing specific user
workflows. • Upgrade to OGSA/GT3.0• Explore Gridlab/Gridsphere,
portlets.
Soon:visportal.nersc.gov
References• The anatomy of the Grid: Enabling Scalable Virtual Organizations, Intl. J.
Supercomputer Applications, 2001.• The Grid Portal Development Kit, J.Novotny, Cuncurrency: Pract. Exper.
2000; 00:1-7• How the Grid will affect the Architecure of Future Visualization Systems,
J.Shalf and W. Bethel, http://www-vis.lbl.gov/Publications• Building a Portal Using GPDK: A Developers Tutorial, J. Novotny,
http://doesciencegrid.org//public/events/GPDW/slides/gpdk-dev.pdf• Globus, A Metacomputing Infrastructure Toolkit. I. Foster, C.
Kessselman, Internation Journal of Supercomputing applications, 1997.• Deploying Web-based Visual Exploratin Tools on the Grid. T.J.Jankun-
Kelly, O. Kreylos, J. Shalf, K-L. Ma, B. Hammann, K. Joy, E.W. Bethel. IEEE Computer Graphics and Applications, march/april 2003.
• VisPortal, Deploying grid-enabled visualization tools through a web-portal interface. http://www-library.lbl.gov/library/docs/LBNL/529/40/PDF/LBNL-52940.pdf
• Globus: www.globus.org• GPDK: www.doesciencegrid.org