Upload
julia-hammond
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Economics Paradigm for “Resource Management and
Scheduling” for Service Oriented P2P/Grid
Computing
Rajkumar Buyya
Melbourne, Australiawww.buyya.com/ecogrid
WW Grid
2
3
Need Honest Answers!
I want to have access to your Grid resources & want to knowhow many of you are willing to give me access ? (following cases)
I am unable to give you access our Australian machines, but I want to have access to yours! [social]
Want to solve academic problems Want to solve business problems
I am willing to gift you Kangaroos! [bartering] I am willing to give you access to my machines, if you
want. (sharing, but no measure & no QoS) [bartering] I am willing to pay you dollars on usage basis.
[economic incentive, market-based, and QoS]
WW Grid
4
Overview
A quick glance at today’s Grid computing Resource Management challenges for next
generation Grid computing A Glance at Approaches to Grid computing. Grid Architecture for Computational Economy Economy Grid = Globus + GRACE Nimrod-G -- Grid Resource Broker Scheduling Experiments Case Study: Drug Design
Application on Grid Conclusions
Scheduling Economics
Grid
EconomyGrid
5
2100
2100 2100 2100 2100
2100 2100 2100 2100
Desktop SMPs or SuperComputers
LocalCluster
GlobalCluster/Grid
PERFORMANCE
Inter PlanetaryGrid!
•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe
Administrative Barriers
EnterpriseCluster/Grid
?
Scalable HPC: Breaking Administrative Barriers & new challenges
6
Why Grids? Large Scale Explorations need them—Killer
Applications. Solving grand challenge applications using
modeling, simulation and analysis
Life Sciences
CAD/CAM
Aerospace
Military ApplicationsDigital Biology Military ApplicationsMilitary Applications
Internet & Ecommerce
7
8
What is Grid ?
An infrastructure that logically couples distributed resources:
Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;
Software – e.g., ASPs renting expensive special purpose applications on demand;
Catalogued data and databases – e.g. transparent access to human genome database;
Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy.
People/collaborators. and presents them as an integrated global resource. It enables the creation of virtual enterprises (VEs)
for resource sharing.
Widearea
data archives
9
P2P/Grid Applications-Drivers
Distributed HPC (Supercomputing): Computational science.
High-Capacity/Throughput Computing: Large scale simulation/chip design & parameter studies.
Content Sharing (free or paid) Sharing digital contents among peers (e.g., Napster)
Remote software access/renting services: Application service provides (ASPs) & Web services.
Data-intensive computing: Virtual Drug Design, Particle Physics, Stock Prediction...
On-demand, realtime computing: Medical instrumentation & Mission Critical.
Collaborative Computing: Collaborative design, Data exploration, education.
Service Oriented Computing (SOC): Computing as Utility: New paradigm and new industries.
10
Building and Using Grids require
Services that make our systems Grid Ready! Security mechanisms that permit resources
to be accessed only by authorized users. (New) programming tools that make our
applications Grid Ready!. Tools that can translate the requirements of
an application/user into the requirements of computers, networks, and storage.
Tools that perform resource discovery, trading, selection/allocation, scheduling and distribution of jobs and collects results.
Globus
?
11
Players in Grid Computing
12
What users want ?Users in Grid Economy &
Strategy Grid Consumers
Execute jobs for solving varying problem size and complexity
Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost
Strategy: minimise expenses Grid Providers
Contribute “idle” resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity
Strategy: maximise return on investment
Challenges for Next Generation Grid
Technology Development
14
Challenges for Grid Computing
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Development Tools
15
Sources of Complexity in Resource Management for World Wide Grid
Computing Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstations, clusters, and
supercomputers, instruments, databases, software) Heterogeneity of fabric management systems (single system image OS,
queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of application requirements (CPU, I/O, memory, and/or
network intensive) Heterogeneity in resource demand patterns (peak, off-peak, ...) Applications need different QoS at different times (time critical results). The
utility of experimental results varies from time to time. Geographical distribution of users & located different time zones Differing goals (producers and consumers have different objectives and
strategies) Unsecure and Unreliable environment
16
Traditional approaches to resource management & scheduling are NOT useful
for Grid ? They use centralised policy that need
complete state-information and common fabric management policy or decentralised consensus-based
policy. Due to too many heterogenous parameters in the Grid it is impossible to
define/get: system-wide performance matrix and common fabric management policy that is acceptable to all.
“Economics” paradigm proved to effective institution in managing decentralization and heterogeneity that is present in human economies!
Fall of USSR & Emergence of US as world superpower! (monopoly?) So, we propose/advocate the use of computational economics principles
in management of resources and scheduling computations on world wide Grid.
Think locally and act globally approach to grid computing!
17
Benefits of Computational Economies
It provides a nice paradigm for managing self interested and self-regulating entities (resource owners and consumers)
Helps in regulating supply-and-demand for resources. Services can be priced in such a way that equilibrium is maintained.
User-centric / Utility driven: Value for money! Scalable:
No need of central coordinator (during negotiation) Resources(sellers) and also Users(buyers) can make their own decisions and try to
maximize utility and profit. Adaptable It helps in offering different QoS (quality of services) to different applications
depending the value users place on them. It improves the utilisation of resources It offers incentive for resource owners for being part of the grid! It offers incentive for resource consumers for being good citizens There is large body of proven Economic principles and techniques available, we can
easily leverage it.
18
New challenges of Computational Economy
Resource Owners How do I decide prices ? (economic models?) How do I specify them ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …..
Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How I trade between timeframe & cost ? ….
Any tools, traders & brokers available to automate the process ?
19
mix-and-match
Object-oriented
Internet/partial-P2P
Network enabled Solvers
Market/Computational Economy
20
Many Grid Projects & Initiatives
Australia Economy Grid Nimrod-G Virtual Lab Active Sheets DISCWorld ..new coming up
Europe UNICORE MOL Lecce GRB Poland MC Broker EU Data Grid EuroGrid MetaMPI Dutch DAS XW, JaWS and many more...
Japan Ninf DataFarm and many more...
USA Globus Legion Javelin AppLeS NASA IPG Condor Harness NetSolve AccessGrid GrADS and many more...
Cycle Stealing & .com Initiatives Distributed.net SETI@Home, …. Entropia, UD, Parabon,….
Public Forums Global Grid Forum P2P Working Group IEEE TFCC Grid & CCGrid conferences
http://www.gridcomputing.com
21
Many Testbeds ? & who pays ?,
who regulates supply and demand ?
GUSTO (decommissioned)
Legion Testbed
NASA IPG
World Wide Grid
WW Grid
22
Testbeds so far -- observations
Who contributed resources & why ? Volunteers: for fun, challenge, fame, charismatic apps, public
good like distributed.net & SETI@Home projects. Collaborators: sharing resources while developing new
technologies of common interest – Globus, Legion, Ninf, Ninf, MC Broker, Lecce GRB,... Unless you know lab. leaders, it is impossible to get access!
How long ? Short term: excitement is lost, too much of admin. Overhead
(Globus inst+), no incentive, policy change,… What we need ? Grid Marketplace!
Regulates supply-and-demand, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.
23
Building an Economy Grid(Next Generation Grid
Computing!)
To enable the creation and promotion of:Grid Marketplace (competitive)
ASPService Oriented Computing
. . .And let users focus on their own work (science, engineering, or commerce)!
24
Grid Node N
GRACE: A ReferenceGrid Architecture for Computational Economy
Grid User
Application
Grid Resource Broker
Grid Service Providers
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Misc. services
Information Server(s)
R2 Rm…
Pricing Algorithms
Accounting
Grid Node1
…
Grid Middleware Services
…
…
HealthMonitor
Grid Market Services
JobExec
Info ?
Secure
Trading
QoS
Storage
Sign-on
Grid Bank
See PDPTA 2000 paper!
25
Economic Models
Price-based: Supply,demand,value, wealth of economic system
Commodity Market Model Posted Price Model Bargaining Model Tendering (Contract Net) Model Auction Model
English, first-price sealed-bid, second-price sealed-bid (Vickrey), and Dutch (consumer:low,high,rate; producer:high, low, rate)
Proportional Resource Sharing Model Monopoly (one provider) and Oligopoly (few players)
consumers may not have any influence on prices. Bartering
Shareholder Model Partnership Model
See SPIE ITCom 2001 paper!: with Heinz Stockinger, CERN!
26
Grid Open Trading Protocols
Get Connected
Call for Bid(DT)
Reply to Bid (DT)
Negotiate Deal(DT)
Confirm Deal(DT, Y/N)
….
Cancel Deal(DT)
Change Deal(DT)
Get Disconnected
Trade Manager Trade Server
Pricing Rules
DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period
API
27
GridFabric
GridApps.
GridMiddleware
GridTools
Networked Resources across Organisations
Computers Clusters Data Sources Scientific InstrumentsStorage Systems
Local Resource Managers
Operating Systems Queuing Systems TCP/IP & UDP
…
Libraries & App Kernels …
Distributed Resources Coupling Services
Security Information … QoSProcess
Development Environments and Tools
Languages Libraries Debuggers … Web toolsResource BrokersMonitoring
Applications and Portals
Prob. Solving Env.Scientific …CollaborationEngineering Web enabled Apps
Resource Trading
Grid Components
Market Info
28
Economy Grid = Globus + GRACE
Applications
MDS
GRAMGlobus Security Interface
Heartbeat MonitorNexus
Local Services
LSF
Condor GRD QBank
PBS
TCP
SolarisIrixLinux
UDP
High-level Services and Tools
DUROC globusrunMPI-G Nimrod/GCC++
Grid Status
GASS
GRACE-TS
GARA
GridFabric
GridApps.
GridMiddleware
GridTools
GBankGMD
eCash
JVM
DUROC
Core Services
Science
Engineering Commerce Portals ActiveSheet……
See IPDPS HWC 2001 paper!
……
29
GRACE components
A resource broker (e.g., Nimrod/G) Various resource trading protocols for different
economic models A mediator for negotiating between users and
grid service providers (Grid Market Directory) A deal template for specifying resource
requirements and services offers Grid Trading Server Pricing policy specification Accounting (e.g., QBank) and payment
management (GridBank, not yet implemented)
30
Pricing, Accounting, Allocations and Job Scheduling Flow @ each site/Grid Level
QBankQBank
Resource Manager44
IBM-LL/PBS/….
00
55 88
66 77
Compute Resourcesclusters/SGI/SP/...
0. Make Deposits, Transfers, Refunds, Queries/Reports1. Clients negotiates for access cost.2. Negotiation is performed per owner defined policies. 3. If client is happy, TS informs QB about access deal.4. Job is Submitted5. Check with QB for “go ahead”6. Job Starts7. Job Completes8. Inform QB about resource resource utilization.
Trade Server 3311
Pricing PolicyPricing Policy22
DB@Each SiteDB@Each Site
GRID BankGRID Bank(digital transactions)(digital transactions)00
31
Service Items to be Charged
CPU - User and System time Memory:
maximum resident set size - page size amount of memory used page faults: with/without physical I/O
Storage: size, r/w/block IO operations Network: msgs sent/received Signals received, context switches Software and Libraries accessed Data Sources (e.g. Protein Data Bank)
32
How to decide Price ?
Fixed price model (like today’s Internet) Dynamic/Demand and Supply (like tomorrow’s Internet) Usage Period Loyalty of Customers (like Airlines favoring frequent flyers!) Historical data Advance Agreement (high discount for corporations) Usage Timing (peak, off-peak, lunch time) Calendar based (holiday/vacation period) Bulk Purchase (register 100 .com domains at once!) Voting -- trade unions decide pricing structure Resource capability as benchmarked in the market! Academic R&D/public-good application users can be offered at
cheaper rate compared to commercial use. Customer Type – Quality or price sensitive buyers. Can be Prescribed by Regulating (Govt.) authorities
33
Payments- Options & Automation
Buy credits in advance / GSPs bill the user later--”pay as you go”
Pay by Electronic Currency via Grid Bank NetCash (anonymity), NetCheque, and Paypal NetCheque: - http://www.isi.edu/gost/info/netcash/
Users register with NC accounting servers, can write electronic cheques and send (e.g email). When deposited, balance is transferred from sender to receiver account.
NetCash - http://www.isi.edu/gost/info/netcheque/ It supports anonymity and it uses the NetCheque system to
clear payments between currency servers. Paypal.com– account+email is linked to credit card.
Enter the recipient’s email address and the amount you wish to request.
The recipient gets an email notification and pays you at www.PayPal.com
Nimrod-G:The Grid Resource Broker
Soft Deadline and Budget-based Economy Grid Resource Broker
for Parameter Processing on P2P Grids
35
Parametric Computing(What Users think of Nimrod
Power)
Multiple RunsSame ProgramMultiple Data Killer Application for the Grid!
ParametersAge Hair
23 CleanAge Hair
23 Clean23 Beard28 Goatee
Age Hair23 Clean23 Beard
Age Hair23 Clean23 Beard28 Goatee28 Clean
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean
Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean
-4000000 Too much
Courtesy: Anand Natrajan, University of Virginia
Magic Engine
See IPDPS 2000 paper!
36
P-study Applications -- Characteristics
Code (Single Program: sequential or threaded)
High Resource Requirements Long-running Instances Numerous Instances (Multiple Data) High Computation-to-Communication
Ratio Embarrassingly/Pleasantly Parallel
37
Sample P-Sweep ApplicationsSample P-Sweep Applications
Bioinformatics: Bioinformatics: Drug Design / Protein Drug Design / Protein
ModellingModelling
SensitivitySensitivityexperiments experiments
on smog formationon smog formation
Combinatorial Combinatorial Optimization:Optimization:
Meta-heuristic Meta-heuristic parameter estimationparameter estimation
Ecological Modelling: Ecological Modelling: Control Strategies Control Strategies
for Cattle Tickfor Cattle Tick
Electronic CAD: Electronic CAD: Field Programmable Field Programmable
Gate ArraysGate ArraysComputer Graphics: Computer Graphics: Ray TracingRay Tracing
High Energy High Energy Physics: Physics:
Searching for Searching for Rare EventsRare Events
Finance: Finance: Investment Risk AnalysisInvestment Risk Analysis
VLSI Design: VLSI Design: SPICE SimulationsSPICE Simulations
Aerospace: Aerospace: Wing DesignWing Design
Network SimulationNetwork SimulationAutomobile:Automobile:
Crash Simulation Crash Simulation
Data MiningData Mining
Civil Engineering:Civil Engineering:Building Design Building Design
astrophysics astrophysics
38
Thesis
Perform parameter sweep (bag of tasks) (utilising distributed resources) within “T” hours or early and cost not exceeding $M.
Three Options/Solutions: Using pure Globus commands Build your own Distributed App & Scheduler Use Nimrod-G (Resource Broker)
39
Remote Execution Steps
Choose Resource
Transfer Input Files
Set Environment
Start Process
Pass Arguments
Monitor Progress
Read/Write Intermediate Files
Transfer Output Files
Summary ViewJob ViewEvent View
+Resource Discovery, Trading, Scheduling, Predictions, Rescheduling, ...
40
Using Pure Globus commands
Do all yourself! (manually)
Total Cost:$???
41
Build Distributed Application & Scheduler
Build App case by case basisComplicated Construction
E.g., AppLeS/MPI based Total Cost:$???
42
Use Nimrod-G
Aggregate Job SubmissionAggregate View
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
South
Submit & Play!
43
Nimrod & Associated Family of Tools
P-sweep App. Composition: Nimrod/
EnfusionResource Management and Scheduling:
Nimrod-G BrokerDesign Optimisations:
Nimrod-OApp. Composition and Online Visualization:
Active SheetsGrid Simulation in Java:
GridSimDrug Design on Grid:
Virtual Lab
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
South
Remote Execution Server(on demand Nimrod Agent)
File Transfer Server
Upcoming?: HEPGrid (+U. Melbourne), GAVE(+Rutherford Appleton Lab)Grid (Un)Aware Virtual Engineering
44
A resource broker for managing, steering, and executing task farming (parametric sweep/SPMD model) applications on Grid based on deadline and computational economy.
Based on users’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability.
Key Features A single window to manage & control experiment Persistent and Programmable Task Farming Engine Resource Discovery Resource Trading Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & results Steering & data management Accounting
Nimrod/G : A Grid Resource Broker
45
A Glance at Nimrod-G Broker
Grid Middleware
Nimrod/G Client Nimrod/G ClientNimrod/G Client
Grid Information Server(s)
Schedule Advisor
Trading Manager
Nimrod/G Engine
GridStore
Grid Explorer
GE GISTM TS
RM & TS
Grid Dispatcher
RM: Local Resource Manager, TS: Trade Server
Globus, Legion, Condor, etc.
G
G
CL
Globus enabled node.Legion enabled node.
GL
Condor enabled node.
RM & TSRM & TS
C LSee HPCAsia 2000 paper!
46
Globus Legion
Fabric
Nimrod Broker
Nimrod ClientsP-Tools (GUI/Scripting)(parameter_modeling)
Legacy Applications
P2P GTS
Farming Engine
Dispatcher & Actuators
Schedule Advisor
Trading Manager
Grid Explorer
Customised Apps(Active Sheet)
Monitoring and Steering Portals
Algorithm1
AlgorithmN
Middleware
. . .
Computers Storage Networks InstrumentsLocal Schedulers
G-Bank. . .
Agents
Resources
Programmable Entities Management
Jobs Tasks
. . .
AgentScheduler JobServer
PC/WS/Clusters Radio TelescopeCondor/LL/Mosix/ . . .Database
Meta-Scheduler
Nimrod/G Grid Broker Architecture
Globus-A
Channels
Legion-A P2P-A. . .
Database(Postgres)
XML
Condor GMD
XML?
IP hourglass ?
47
A Nimrod/G Monitor
A Nimrod/G Monitor
CostCostDeadlineDeadline
Legion hosts
Globus Hosts
Bezek is in both Globus and Legion Domains
Arlington
Alexandria
Richmond
HamptonNorfolk
Virginia BeachChesapeakePortsmouth
Newport News
Roanoke
Ap p om a toxRive r
Ja m esRive r
Shena nd oa hRive r
Ra p p a ha nnoc kRive r
Potom a cRive r
VIRGINIA77
81
64
64
66
85
48
User Requirements: Deadline/Budget User Requirements: Deadline/Budget
49
Active Sheet: Spreadsheet Processing on Grid
NimrodNimrodProxyProxy
Nimrod/GNimrod/G
See HPC 2001 paper!
50
51
Nimrod/G Interactions
Grid Infoservers
Resource Discovery
QueuingSystem
Processserver
Resource allocation
(local)
Userprocess
File accessI/Oserver
Gatekeeper node
NimrodAgent
Computational node
Dispatcher
Root node
Scheduler
FarmingEngine
Grid Trade Server
“Do this in 30min. for $10?”
52
Discover Discover ResourcesResources
Distribute JobsDistribute Jobs
Establish Establish RatesRates
Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?
Evaluate & Evaluate & RescheduleReschedule
Discover Discover More More
ResourcesResources
Adaptive SchedulingAlgorithms
Execution Time (not beyond deadline)
Execution Cost (not beyond budget)
Time Minimisation Minimise Limited by budgetCost Minimisation Limited by deadline MinimiseNone Minimisation Limited by deadline Limited by budget
Adaptive Scheduling Algorithms
Compose & Compose & ScheduleSchedule
See HPDC AMS 2001 paper!
53
Cost Model
Without cost ANY shared system becomes un-managable
Charge users more for remote facilities than their own
Choose cheaper resources before more expensive ones
Cost units (G$) may be Dollars Shares in global facility Stored in bank
54
Cost Matrix @ Grid site X
Non-uniform costing Encourages use of
local resources first Real accounting
system can control machine usage
11 33
22 11User 5User 5
Mach
ine 1
Mach
ine 1
User 1User 1
Mach
ine 5
Mach
ine 5
Resource Cost = Function (cpu, memory, disk, network, software, QoS, current demand, etc.)
Simple: price based on peaktime, offpeak, discount when less demand, ..
55
Deadline and Budget-based Cost Minimization Scheduling
1. Sort resources by increasing cost.2. For each resource in order, assign as
many jobs as possible to the resource, without exceeding the deadline.
3. Repeat all steps until all jobs are processed.
56
M - Resources, N - Jobs, D - deadline Note: Cost of any Ri is less than any of Ri+1 …. Or Rm
RL: Resource List need to be maintained in increasing order of cost Ct - Time when accessed (Time now) Ti - Job runtime (average) on Resource i (Ri) [updated periodically]
Ti is acts as a load profiling parameter. Ai - number of jobs assigned to Ri , where:
Ai = Min (No.Unassigned Jobs, No. Jobs Ri can complete by remaining deadline) No.UnAssignedJobsi = Diff( N, (A1+…+Ai-1)) JobsRi consume = RemainingTime (D- Ct) DIV Ti
ALG: Invoke Job Assignment() periodically until all jobs done. Job Assignment()/Reassignment():
Establish ( RL, Ct , Ti , Ai ) dynamically – Resource Discovery. For all resources (I = 1 to M) { Assign Ai Jobs to Ri , if required}
Deadline-based Cost-minimization Scheduling
57
Deadline and Budget Constraint (DBC) Time Minimization
Scheduling
1. For each resource, calculate the next completion time for an assigned job, taking into account previously assigned jobs.
2. Sort resources by next completion time.3. Assign one job to the first resource for
which the cost per job is less than the remaining budget per job.
4. Repeat all steps until all jobs are processed. (This is performed periodically or at each scheduling-event.)
58
Deadline and Budget Constraint (DBC) Time+Cost Min. Scheduling
1. Split resources by whether cost per job is less than budget per job.
2. For the cheaper resources, assign jobs in inverse proportion to the job completion time (e.g. a resource with completion time = 5 gets twice as many jobs as a resource with completion time = 10).
3. For the dearer resources, repeat all steps (with a recalculated budget per job) until all jobs are assigned.
4. [Schedule/Reschedule] Repeat all steps until all jobs are processed.
Evaluation of Scheduling Heuristics
A Hypothetical Application on
World Wide Grid
WW Grid
60
Globus+LegionGRACE_TS
Australia
Monash Uni.:
Linux cluster
Solaris WS
Nimrod/G
Globus +GRACE_TS
Europe
ZIB/FUB: T3E/Mosix Cardiff: Sun E6500Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCalabria: Cluster CERN: ClusterPozman: SGI/SP2
Globus +GRACE_TS
Asia/Japan
Tokyo I-Tech.:ETL, Tuskuba
Linux cluster
Globus/LegionGRACE_TS
North America
ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux ClusterUD: Linux clusterUTK: Linux cluster
Internet
World Wide Grid (WWG)
Globus +GRACE_TS South America
Chile: Cluster
WW Grid
WW Grid
61
Experiment-1 Setup
Workload: 165 jobs, each need 5 minute of cpu time
Deadline: 1 hrs. and budget: 800,000 units
Strategy: minimise cost and meet deadline
Execution Cost with cost optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155 (G$)
62
Resources Selected & Price/CPU-sec.
Resource Type & Size
Owner and Location
Grid services
Peaktime Cost (G$)
Offpeak cost
Linux cluster (60 nodes)
Monash, Australia
Globus/Condor 20 5
IBM SP2 (80 nodes)
ANL, Chicago, US
Globus/LL 5 10
Sun (8 nodes) ANL, Chicago, US
Globus/Fork 5 10
SGI (96 nodes) ANL, Chicago, US
Globus/Condor-G
15 15
SGI (10 nodes) ISI, LA, US Globus/Fork 10 20
63
Execution @ AU Peak Time
0
2
4
6
8
10
12
Time (minutes)
Jo
bs
Linux clus ter - Monash (20) Sun - ANL (5) SP2 - ANL (5) SGI - ANL (15) SGI - ISI (10)
64
Execution @ AU Offpeak Time
0
2
4
6
8
10
12
Time (minutes)
Jo
bs
Linux clus ter - Monash (5) Sun - ANL (10) SP2 - ANL (10) SGI - ANL (15) SGI - ISI (20)
65
AU peak: Resources/Cost in Use
0
50
100
150
200
250
300
350
400
450
500
Tim e (in m in.)
Co
st o
f R
eso
urc
es in
Use
0
5
10
15
20
25
30
35
40
Tim e (in m in.)
Res
ou
rces
(N
o. o
f C
PU
s) in
Use
After the calibration phase, note the difference in pattern of two graphs. This is when scheduler stopped using
expensive resources.
66
AU offpeak: Resources/Cost in Use
0
50
100
150
200
250
300
350
Time (in min.)
Co
st o
f R
eso
urc
es i
n U
se
0
5
10
15
20
25
30
Time (in min.)
Res
ou
rces
(N
o.
of
CP
Us)
in
Use
67
Experiment-2 Setup
Workload: 165 jobs, each need 5 minute of CPU time
Deadline: 2 hrs. and budget: 396000 units Strategy: minimise time / cost Execution Cost with cost optimisation
Optimise Cost: 115200 (G$) (finished in 2hrs.) Optimise Time: 237000 (G$) (finished in 1.25 hr.) In this experiment: Time-optimised scheduling run
costs double that of Cost-optimised. Users can now trade-off between Time Vs. Cost.
68
Resources Selected & Price/CPU-sec.
Resource & Location
Grid services & Fabric
Cost/CPU sec.or unit
No. of Jobs Executed
Time_Opt Cost_Opt.
Linux Cluster-Monash, Melbourne, Australia
Globus, GTS, Condor
2 64 153
Linux-Prosecco-CNR, Pisa, Italy
Globus, GTS, Fork 3 7 1
Linux-Barbera-CNR, Pisa, Italy
Globus, GTS, Fork 4 6 1
Solaris/Ultas2
TITech, Tokyo, Japan
Globus, GTS, Fork 3 9 1
SGI-ISI, LA, US Globus, GTS, Fork 8 37 5
Sun-ANL, Chicago,US Globus, GTS, Fork 7 42 4Total Experiment Cost (G$) 237000 115200
Time to Complete Exp. (Min.) 70 119
69
DBC Scheduling for Time Optimization
0
2
4
6
8
10
12
Time (in Minute)
No.
of
Tas
ks i
n E
xecu
tion
Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR
Solaris /Ultas2-TITech SGI-ISI Sun-ANL
70
DBC Scheduling for Cost Optimization
0
2
4
6
8
10
12
14
Time (in Minute)
No.
of
Tas
ks i
n E
xecu
tion
Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR
Solaris /Ultas2-TITech SGI-ISI Sun-ANL
Application Case Study
The Virtual Laboratory Project: "Molecular Modelling for Drug Design" on Peer-to-Peer Grid
72
Virtual Drug Design: Data Intensive Computing on Grid
A Virtual Laboratory for “Molecular Modelling for Drug Design” on Peer-to-Peer Grid.
It provides tools for examining millions of chemical compounds (molecules) in the Protein Data Bank (PDB) to identify those having potential use in drug design.
In collaboration with: Kim Branson, Structural
Biology, Walter and Eliza Hall Institute (WEHI)
http://www.csse.monash.edu.au/~rajkumar/vlab
73
Virtual Drug DesignA Virtual Lab for “Molecular Modeling for Drug Design” on P2P Grid
“Screen 2K molecules in 30min. for $10”
Grid Market Directory
ResourceBroker
Grid Info. Service
GTS
GTS
GTS
GTS
“Give me list PDBs sourcesOf type aldrich_300?”
“serv
ice co
st?”
(GTS - Grid Trade Server)
PDB2
“get mol.10 from pdb1 & screen it.”
Data Replica Catalogue
“service providers?”
GTS
PDB1
“mol.10 please?”
“mol.5 please?”
(RB maps suitable Grid nodes and Protein DataBank)
74
DataGrid Brokering
Nimrod/GComputational
Grid Broker
Data Replica CataloguePDB Broker
Algorithm1
AlgorithmN
. . .
PDB Service
PDB2
“Screen mol.5 please?”
GSP1 GSP2 GSP4GSP3(Grid Service Provider)
GSPm
PDB Service
GSPn
1
“advise PDB source?
2“selection & advise: use GSP4!”
5Grid Info. Service
3
“Is GSP4 healthy?”
4
“mol.5 please?”6
“PDB replicas please?”
“Screen 2K molecules in 30min. for $10”
7
“process & send results”
75
Software Tools
Molecular Modelling Tools (DOCK) Parameter Modelling Tools (Nimrod/enFusion) Grid Resource Broker (Nimrod-G) Data Grid Broker Protein Data Bank (PDB) Management and Intelligent Access
Tools PDB databse Lookup/Index Table Generation. PDB and associated index-table Replication. PDB Replica Catalogue (that helps in Resource Discovery). PDB Servers (that serve PDB clients requests). PDB Brokering (Replica Selection). PDB Clients for fetching Molecule Record (Data Movement).
Grid Middleware (Globus and GrACE) Grid Fabric Management (Fork/LSF/Condor/Codine/…)
76
DOCK code*(Enhanced by WEHI, U of
Melbourne)
A program to evaluate the chemical and geometric complementarities between a small molecule and a macromolecular binding site.
It explores ways in which two molecules, such as a drug and an enzyme or protein receptor, might fit together.
Compounds which dock to each other well, like pieces of a three-dimensional jigsaw puzzle, have the potential to bind.
So, why is it important to able to identify small molecules which may bind to a target macromolecule?
A compound which binds to a biological macromolecule may inhibit its function, and thus act as a drug.
Thus disabling the ability of (HIV) virus attaching itself to molecule/protein!
With system specific code changed, we have been able to compile it for Sun-Solaris, PC Linux, SGI IRIX, Compaq Alpha/OSF1
* Original Code: University of California, San Francisco: http://www.cmpharm.ucsf.edu/kuntz/
77
Dock input filescore_ligand yesminimize_ligand yesmultiple_ligands norandom_seed 7anchor_search notorsion_drive yesclash_overlap 0.5conformation_cutoff_factor 3torsion_minimize yesmatch_receptor_sites norandom_search yes . . . . . . . . . . . .maximum_cycles 1ligand_atom_file S_1.mol2receptor_site_file ece.sphscore_grid_prefix ecevdw_definition_file parameter/vdw.defnchemical_definition_file parameter/chem.defnchemical_score_file parameter/chem_score.tblflex_definition_file parameter/flex.defnflex_drive_file parameter/flex_drive.tblligand_contact_file dock_cnt.mol2ligand_chemical_file dock_chm.mol2ligand_energy_file dock_nrg.mol2
Molecule to Molecule to be screenedbe screened
78
score_ligand $score_ligandminimize_ligand $minimize_ligandmultiple_ligands $multiple_ligandsrandom_seed $random_seedanchor_search $anchor_searchtorsion_drive $torsion_driveclash_overlap $clash_overlapconformation_cutoff_factor $conformation_cutoff_factortorsion_minimize $torsion_minimizematch_receptor_sites $match_receptor_sitesrandom_search $random_search . . . . . . . . . . . .maximum_cycles $maximum_cyclesligand_atom_file ${ligand_number}.mol2receptor_site_file $HOME/dock_inputs/${receptor_site_file}score_grid_prefix $HOME/dock_inputs/${score_grid_prefix}vdw_definition_file vdw.defnchemical_definition_file chem.defnchemical_score_file chem_score.tblflex_definition_file flex.defnflex_drive_file flex_drive.tblligand_contact_file dock_cnt.mol2ligand_chemical_file dock_chm.mol2ligand_energy_file dock_nrg.mol2
Parameterized Dock input file
Molecule to be Molecule to be screenedscreened
79
parameter database_name label "database_name" text select oneof "aldrich" "maybridge" "maybridge_300" "asinex_egc" "asinex_epc" "asinex_pre" "available_chemicals_directory" "inter_bioscreen_s" "inter_bioscreen_n" "inter_bioscreen_n_300" "inter_bioscreen_n_500" "biomolecular_research_institute" "molecular_science" "molecular_diversity_preservation" "national_cancer_institute" "IGF_HITS" "aldrich_300" "molecular_science_500" "APP" "ECE" default "aldrich_300";
parameter score_ligand text default "yes";parameter minimize_ligand text default "yes";parameter multiple_ligands text default "no";parameter random_seed integer default 7;parameter anchor_search text default "no";parameter torsion_drive text default "yes";parameter clash_overlap float default 0.5;parameter conformation_cutoff_factor integer default 5;parameter torsion_minimize text default "yes";parameter match_receptor_sites text default "no";parameter random_search text default "yes"; . . . . . . . . . . . .parameter maximum_cycles integer default 1;parameter receptor_site_file text default "ece.sph";parameter score_grid_prefix text default "ece";parameter ligand_number integer range from 1 to 200 step 1;
Dock PlanFile (contd.)
Molecules to be Molecules to be screenedscreened
80
task nodestart copy ./parameter/vdw.defn node:. copy ./parameter/chem.defn node:. copy ./parameter/chem_score.tbl node:. copy ./parameter/flex.defn node:. copy ./parameter/flex_drive.tbl node:. copy ./dock_inputs/get_molecule node:. copy ./dock_inputs/dock_base node:.endtasktask main node:substitute dock_base dock_run node:substitute get_molecule get_molecule_fetch node:execute sh ./get_molecule_fetch node:execute $HOME/bin/dock.$OS -i dock_run -o dock_out copy node:dock_out ./results/dock_out.$jobname copy node:dock_cnt.mol2 ./results/dock_cnt.mol2.$jobname copy node:dock_chm.mol2 ./results/dock_chm.mol2.$jobname copy node:dock_nrg.mol2 ./results/dock_nrg.mol2.$jobnameendtask
Dock PlanFile
81
Nimrod/TurboLinux enFuzion GUI tools for Parameter Modeling
82
Docking Experiment Preparation
Setup PDB DataGrid Index PDB databases Pre-stage (all) Protein Data Bank (PDB) on replica sites Start PDB Server
Create Docking GridScore (receptor surface details) for a given receptor on home node.
Pre-Staging Large Files required for Docking: Pre-stage Dock executables and PDB access client on Grid nodes, if
required (e.g., dock.Linux, dock.SunOS, dock.IRIX64, and dock.OSF1 on Linux, Sun, SGI, and Compaq machines respectively). Use globus-rcp.
Pre-stage/Cache all data files (~3-13MB each) representing receptor details on Grid nodes.
This can can be done demand by Nimrod/G for each job, but few input files are too large and they are required for all jobs). So, pre-staging/caching at http-cache or broker level is necessary to avoid the overhead of copying the same input files again and again!
83
Protein Data Bank
Databases consist of small molecules from commercially available organic synthesis libraries, and natural product databases.
There is also the ability to screen virtual combinatorial databases, in their entirety.
This methodology allows only the required compounds to be subjected to physical screening and/or synthesis reducing both time and expense.
84
Target Testcase
The target for the test case: electrocardiogram (ECE) endothelin converting enzyme. This is involved in “heart stroke” and other transient ischemia.
Is·che·mi·a : A decrease in the blood supply to a bodily organ, tissue, or part caused by constriction or obstruction of the blood vessels.
85
Nimrod/G in Action:Screening on World-Wide
Grid
86
Any Scientific Discovery ? Did your collaborator invent new drug for
xxxx?
Not Yet
Anyway, checkout the announcement of Nobel-
prize winners for next year
?
Conclude with a comparison with the Electrical
Grid………..
Where we are ????
Courtesy: Domenico Laforenza
Alessandro Volta in Paris in 1801 inside French National Institute shows the battery
while in the presence of Napoleon I
Fresco by N. Cianfanelli (1841) (Zoological Section "La Specula" of National History Museum of Florence
University)
89
….and in the future, I imagine a worldwidePower (Electrical) Grid …...
What ?!?!This is a mad man…
Oh, monDieu !
90
2001 - 1801 = 200 Years
91
” I think there is a world market for about five computers.”Thomas J. Watson Sr., IBM Founder, 1943
Can we Predict its Future ?
92
Summary and Conclusions P2P and Grid Computing is emerging as a next generation
computing platform for solving large scale problems through sharing of geographically distributed resources.
Resource management is a complex undertaking as systems need to be adaptive, scalable, competitive,…, and driven by QoS.
We proposed a framework based on “computational economies” and discussed several economic models for resource allocation and for regulating supply-and-demand for resources.
Scheduling experiments on World Wide Grid demonstrate our Nimrod-G broker ability to dynamically lease or rent services at runtime based on their quality, cost, and availability depending on consumers QoS requirements.
Economics paradigm for QoS driven resource management is essential to push P2P/Grids into mainstream computing!
93
Download Software & Information
Nimrod & Parameteric Computing: http://www.csse.monash.edu.au/~davida/nimrod/
Economy Grid & Nimrod/G: http://www.buyya.com/ecogrid/
Virtual Laboratory/Virtual Drug Design: http://www.buyya.com/vlab/
Grid Simulation (GridSim) Toolkit (Java based): http://www.buyya.com/gridsim/
World Wide Grid (WWG) testbed: http://www.buyya.com/ecogrid/wwg/ Looking for new volunteers to grow
Please contact me to barter your & our machines!
Want to build on our work/collaborate: Talk to me now or email: [email protected]
94
Thank You… Any ??
Thank You… Any ??
95
Further Information
Books: High Performance Cluster Computing, V1,
V2, R.Buyya (Ed), Prentice Hall, 1999. The GRID, I. Foster and C. Kesselman (Eds),
Morgan-Kaufmann, 1999. IEEE Task Force on Cluster Computing
http://www.ieeetfcc.org Global Grid Forum
www.gridforum.org
IEEE/ACM CCGrid’xy: www.ccgrid.org CCGrid 2002, Berlin: ccgrid2002.zib.de
Grid workshop - www.gridcomputing.org
96
Further Information
Cluster Computing Info Centre: http://www.buyya.com/cluster/
Grid Computing Info Centre: http://www.gridcomputing.com
IEEE DS Online - Grid Computing area:
http://computer.org/dsonline/gc
Compute Power Market Project http://www.ComputePower.com