1
History and New Developments in State-wide Grid Computing Course
NC A & T UniversitySaturday Oct 18, 2008
Dr. Barry WilkinsonUniversity of North Carolina
Charlotte
Dr. Clayton FernerUniversity of North Carolina
Wilmington
2
Outline
Grid Computing course 2004, 2005 New direction 2007 Grid Computing course 2008 Grid Computing course
3
Grid Computing Course
Undergraduate/graduate Hands-on with distributed grid infrastructure Teleconferencing facilities - students and faculty at
many institutions participating Expert guest speakers near end of course Probably first such course for undergraduate students
and distributed sites using large-scale teleconferencing facilities and a truly distributed grid infrastructure.
Funding from NSF and UNC- OP
4
Grid Computing Course 2004-2005
Western Carolina University
UNC Greensboro
Appalachian State University
UNC AshevilleWinston-Salem State
University
UNC Chapel Hill
NC State University
NC Central University
Lenoir Rhyne College
UNC Wilmington
Elon University
UNC Pembroke
UNC Charlotte
Wake Tech. Comm. College
© World Sites Atlas (sitesatlas.com)
SOUTH CAROLINA
VIRGINIA
TENNESSEE
GEORGIA
NORTH CAROLINA
5
Problem with previous offerings
In 2004 and 2005, course taught from a bottom-up perspective
We started with Web services then moved on the Grid services
Everything was command-line Only toward the end of the semester did we
introduce a workflow editor and schedulers (such as Condor or Sun Grid Engine)
6
2007 Grid Computing Course
Goal: More top-down perspective Started with a Grid portal interface and then did
command-line tasks afterwards so that students could see the “big” picture
Moved some assignments to be done on student’s own computer to eliminate problems with using a centralized server. Required students to install Globus core, and other
packages themselves Course taught as an experimental course in Fall 2007
with UNC-C, UNC-W and UNC-A only.
9
Registration
Students filled out request for account Certificate Request generated and sent to
Certificate Authority automatically
Certificate signed , installed in MyProxy server, email confirmation to student
Student logged onto Portal, acquires a proxy, creates a java program locally, transfers it using portal, and submits job.
10
Registration Activities
New User
PURSe on-line registration form
CA/SystemAdministrator
Create accounts, set access control, sign certificate, …
Fill in formProvide password and other information
Email• Request Confirmation• Acknowledgement
11
Assignment 2 (Using command-line)
Students performed the same tasks as in Assignment 1 but used a command-line
Install ssh client (e.g. putty) if necessary Set up your credentials (grid-cert-request) Email Certificate Authority to sign certificate Install certificate and create proxy Submit various jobs with globusrun-ws Some jobs submitted with xml job description files
12
Assignment 3 (Using Condor)
Create proxy Check the status of the Condor pool Create a test submit description text file Submit jobs and check status Using different Condor universes Still a command-line interface (In 2005, we used Sun Grid Engine, which has
a GUI interface)
13
Assignment 4 (Creating and deploying a Grid Service)
Install GT 4 core and associated software JDK 1.4.2+ Ant 1.5.1+ Python 2.4+ Globus 4.0 core
Testing installation Start container
Create, deploy, and test simple GT4 Grid Services Deplore prewritten
service and test with client
Add Functionality to Service
Need to handle WSDL (XML) and other files
14
Assignment 5 (Using a GridNexus)
Install GridNexus workflow editor
Create Web service workflow
Create Grid service workflow
Submitting a job to GRAM
15
Assignment 6 (Implementing portlets with Gridsphere)
Install Gridsphere and associated software Java 6 SDK ant tomcat 5.5.20
Install Gridsphere First start Tomcat
Create portlets Prewritten odd-even
portlet Installation involves
handling deployment descriptor files etc.
Portlet to add, subtract, multiply and divide two numbers.
17
Assignment 7 (MPI Program)
Write simple MPI program (Matrix Multiplication)
We couldn't do much more than work with “embarrassingly parallel” applications since we only had a few weeks to deal with MPI
The students only executed their programs on one cluster (not really using the Grid).
18
Mini Projects
Teams of 3 members Objective was to create a new Grid assignment Assignment had to involve created a Grid
application with a GUI (such as a workflow or portal)
Assignment had to be written up as though the student would give it to their classmates
Focus on dissemination May create future assignments
19
Mini Projects
Teams had to provide a written report (with solutions)
Teams had to give a presentation
20
Keeping to Assignment Schedules
Each assignment allocated 1-2 weeks to complete. Posted three dates:
1. Date assignment was set
2. Date that students had to report any system problems preventing them from proceeding
3. Date due Fall 2004 many system problems (Globus 3.2) Fall 2005 much fewer problems (Globus 4.0) Spring 2007, no system problems reported
21
Keeping to Assignment Schedules
In 2004 and 2005, students created and deployed their Grid services on the same machine
An error in a service could make the Globus container inoperable for the other students
In 2007, student install the Globus 4 core on an individual PC (lab machine or their own) and used their own container
This turned out to be a much better solution!
22
Lessons Learned Students responded positively to using their own
computers that were under their direct control.
Some students did have problems with their installations Nothing that could not resolved quickly such as not
setting paths or using the wrong version of the software.
Some Grid computing assignments still have to be done on a Grid platform but using personal computers where possible avoided many problems
23
Avoiding Problems
It requires immense work to prepare for a hands-on Grid computing course.
Critical that all assignments are fully tested prior to the start of class and that all computer systems are reliable and the software maintained.
Assignments went much smoother by requiring students to use personal computers when possible.
2008 Grid computing course
New NSF funding 2008-2010
Goal:
To demonstrate that GridNexus workflow editor will provide a better user-interface in the classroom
Participating sites: UNC-C, UNC-W, NC A & T and UNC-A
25
http://www.csc.uncw.edu/~cferner/ITCS4146F08/
Password protected
Arranged so that both instructors can modify page – very important
Course Home Page
Organizational details
We attempt to have both instructors give part of each class period to give variety within the class period.
Needs coordination so we have a standing telephone conversation before each class.
New assignments
Assigments added/modified to show benefits of GridNexus:
Assignment 1 User registration and using Portal
Assignment 2 Command line version of assignment 1
Assignment 3 GridNexus job workflow
Assignment 4 Using Condor, now includes ClassAd matchmaking and DAGMan workflow.
Assignment 5 Creating Globus 4.0 services
Assignment 6 GridNexus service workflow
Assignment 7 Portlet design in GridSphere .
28
Assignment 1
Students write a java program that computes the area of a flower bed (piece-wise function) using the trapezoid method.
Easy program
They use the portal to upload the class file to a UNCC server and run it through the portal
29
Assignment 2
Students write a java program that given the area of a flower bed computes the volume and cost of mulch to cover the bed.
Even easier program
They use the command line to upload the class file to a UNCW server and run it use globusrun-ws
30
Assignment 3
Students create a workflow that
executes the program to compute the area of the flower bed on UNCC server
Transfers the results to the UNCW server
Executes the program to compute the volume and cost of the mulch
Transfer the results back to the UNCC server
Graduate students create a workflow to do the same thing but using an RSL including file staging
31
Assignment 3
Students create a workflow that
executes the program to compute the area of the flower bed on UNCC server
Transfers the results to the UNCW server
Executes the program to compute the volume and cost of the mulch
Transfer the results back to the UNCC server
Graduate students create a workflow to do the same thing but using an RSL including file staging
32
Survey 1
Student were asked to fill out an online survey asking them various questions about the first 3 assignments
They were asked to compare how they would create a workflow moving data between servers using a portal and command line.
33
Plan for Assignment 6
Students will be asked to create a workflow to use Web Services within grid environment using GridNexus.
A simple workflow to work with a service where the resource is within the web service (they share the results)
A workflow where they use a factory to create a new resource
A workflow to perform the mulch problem using multiple servers and moving data
Evaluation plan
• sites (UNC-W and UNC-C)
• site (NC A & T)
• Professional evaluator (Barbara Heath)
UNC-C module development
Release to site
Jan 2008
August 2008
June 2008
UNC-W module development
NCAT site testing
Remote testing
site development and testing
Jan 2009
Second year material development:
August 2009
June 2009
Dec. 2008
Dec. 2009
Feedback Remote
evaluation
Exchange
Develop materials
Develop materials
Test collectively on NCREN
(joint course)
Feedback
Test locally Test locally
Remote evaluation
Exchange
Develop materials
Develop materials
Other institutions
Test locally
Feedback
Feedback